Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Zebras & Lasers (PyBay 2018)

Zebras & Lasers (PyBay 2018)

F932a58935f19875076d7b2232a20fd3?s=128

Jonas Neubert

August 18, 2018
Tweet

More Decks by Jonas Neubert

Other Decks in Programming

Transcript

  1. Zebras and Lasers: A crash course on barcodes with Python

    Jonas Neubert 3rd Annual Regional Python Conference August 16 - 19, 2018 | San Francisco, CA
  2. Jonas Neubert Software Engineer at Zymergen slides: http://jonasneubert.com/talks/pybay2018.html twitter: @jonemo

  3. From First Principles ʵ Barcodes Theory ż Barcodes with Python

    Ʉ Barcodes in The Realworld
  4. None
  5. Barcodes Theory

  6. Symbology - Geometric primitives (lines, circles, matrix, ...) - Arrangement

    of primitives (single row, concentric, stacked, …) - Symbol table (pixels → bits) - Encoding (bits → information) - Checksums and Error Correction Code - What the numbers mean (part numbers, ZIP codes, ...)
  7. Code 128: Full ASCII, variable length, continuous (11,3), three character

    sets
  8. 11 slots 3 bars & 3 gaps per character Maximum

    Bar Size: 4 Slots 106 Different Characters 3 Character Sets for “Full-Ascii” Code 128: Full ASCII, variable length, continuous (11,3), three character sets
  9. character “P” (0x30) Code 128: Full ASCII, variable length, continuous

    (11,3), three character sets
  10. character “H”(0x28) Code 128: Full ASCII, variable length, continuous (11,3),

    three character sets
  11. character “P”(0x30) Code 128: Full ASCII, variable length, continuous (11,3),

    three character sets
  12. “PHP” Code 128: Full ASCII, variable length, continuous (11,3), three

    character sets
  13. “PHP” cHecksUm (0x43) Code 128: Full ASCII, variable length, continuous

    (11,3), three character sets
  14. “PHP” stARt coDE (0x68) STOP Code ..(0x70) cHecksUm (0x43) Code

    128: Full ASCII, variable length, continuous (11,3), three character sets
  15. “PHP” stARt coDE (0x68) STOP Code ..(0x70) cHecksUm (0x43) quiet

    ZonEs Code 128: Full ASCII, variable length, continuous (11,3), three character sets
  16. Code 39: Numbers and uppercase letters (39 chars), discrete 3-of-9

    2-width 12 slots 5 bars & 4 gaps per character Two distinct widths 3 out of 9 Bars or Gaps are 2x wide Variable Gap between characters 39 Different Characters: A-Z 0-9 *$/+%
  17. * Start code ? Just use “*” STOP CODE? Another

    “*” cHecksUm optional * H P P Code 39: Numbers and uppercase letters (39 chars), discrete 3-of-9 2-width
  18. Interleaved 2-of-5: Numbers only, variable length, continuous 2-of-5 2-width

  19. 0 8 8 1 Start Guard StoP Guard cHecksUm optional

    Interleaved 2-of-5: Numbers only, variable length, continuous 2-of-5 2-width
  20. POSTNET: Numbers only, fixed length, clocked continuous 2-of-5, height-modulated USPS

    pre-2009
  21. POSTNET: Numbers only, fixed length, clocked continuous 2-of-5, height-modulated USPS

    pre-2009
  22. Start GUard STOP guarD cHecksUm 9 4 1 4 3

    3 0 0 8 - POSTNET: Numbers only, fixed length, clocked continuous 2-of-5, height-modulated USPS pre-2009
  23. Zielcode: Numbers only, fixed length, continuous 2-of-5 1-width Deutsche Post

    Source: Helmut Schraets: Meilensteine der Briefcodierung (1998). https://www.briefmarken-kevelaer.de/media/files/Meilensteine-Neu.pdf
  24. Intelligent Mail®: Numbers only, variable length, clocked 4-state, height-modulated US

    Postal since 2009 00-300-123456-123456789-94143-3008-00 “Send First Class letter with no services (300) no OEL information (00) to UCSF Conference center (94143-3008-00) using mAILer ID 123456 and sequence number 123456789”
  25. Source: USPS: Intelligent Mail Barcode 4-state Specification (2015), Intelligent Mail®:

    Numbers only, variable length, clocked 4-state, height-modulated US Postal since 2009
  26. :CueCat: Like Code 128 but at 22.5° angle and proprietary

    code
  27. Rule of Thumb: Height = 0.15 * Width Anonymous 1980s

    engineer pulling Number out of his Hat Bacon
  28. Bacon & Eggs Rule of Thumb: Height = 0.15 *

    Width Anonymous 1980s engineer pulling Number out of his Hat
  29. Bacon & Egg with Toast Rule of Thumb: Height =

    0.15 * Width Anonymous 1980s engineer pulling Number out of his Hat
  30. Code 49: Stacked linear codes, fixed length (49 chars) Full

    Breakfast with Bacon & Eggs and Toast & Jam
  31. Code 16k: Stack of inverted Code 128 symbols, 77 chars

    or 154 digits
  32. PDF417: “High capacity”, variable length, stack of continuous (17,4) codes

    Full Breakfast with Bacon, Baked Beans and Eggs and Toast or an English Muffin as a side.
  33. PDF417: “High capacity”, variable length, stack of continuous (17,4) codes

  34. Data Matrix: Arbitrary data, up to 1556 bytes/2335 chars/3116 digits

    PyBay
  35. Data Matrix: Arbitrary data, up to 1556 bytes/2335 chars/3116 digits

    White in Top right means: Uses EC200 PyBay Left and bottom guards are solid Top and right guards are clocked
  36. Data Matrix: Arbitrary data, up to 1556 bytes/2335 chars/3116 digits

    White in Top right means: Uses EC200 PyBay Left and bottom guards are solid Top and right guards are clocked
  37. Data Matrix: Arbitrary data, up to 1556 bytes/2335 chars/3116 digits

    White in Top right means: Uses EC200 PyBay Left and bottom guards are solid Top and right guards are clocked P y B P y a B a REED-SOLOMON ERROR CORRECTION PADDING
  38. Data Matrix: Arbitrary data, up to 1556 bytes/2335 chars/3116 digits

  39. Data Matrix: Arbitrary data, up to 1556 bytes/2335 chars/3116 digits

  40. Semacode: Code for storing URLs in Datamatrix Source: Bernd Hopfengärtner:

    Hello World! (2007) http://hello.w0r1d.net/
  41. QR: up to 7089 digits or 4296 ASCII characters or

    1817 Kanji characters
  42. QR: up to 7089 digits or 4296 ASCII characters or

    1817 Kanji characters Source: Xinhua (2017) http://www.xinhuanet.com/photo/2017-09/14/c_1121662356_2.htm
  43. Aztec: up to 3832 digits or 3067 characters Maxicode: exactly

    596 bits, hexagonal elements Grid Matrix: optimized for encoding Chinese characters Code One: First non-proprietary 2D symbology (1992) Micro QR: Variation of QR Code, numeric, up to 35 digits, single alignment mark ← HIBC Datamatrix: Variation of Datamatrix, up to 36 characters, used in medical devices ←
  44. UPC/EAN: Numbers only, fixed length of 12 or 13 digits,

    continuous (7,2) 4-width Source: Businessweek, April 7, 1973. Available on the website of the ID History Museum at http://idhistory.com/ibm/Busweek1973001.pdf
  45. 1D Interleaved 2-of-5 Code 39 Code 128 :CueCat Bullseye UPC/EAN

    Stacked Code 49 Code 16k PDF417 Matrix Datamatrix Aztec Maxicode Grid Matrix Code One QR Mini QR POSTNET RM4SCC PostBar Intelligent Mail
  46. 1D Interleaved 2-of-5 Code 39 Code 128 :CueCat Codabar Code

    11 Code 93 93i Channel Code Bullseye UPC-A, UPC-E EAN-13, EAN-8 GS1-128 GS1 Databar (RSS) Plessey Code Stacked Code 49 Code 16k PDF417 MicroPDF417 Codablock SuperCode UltraCode GS1 Databar St. Matrix Datamatrix Aztec Maxicode Grid Matrix Code One QR Mini QR Vericode MiniCode GoCode POSTNET RM4SCC PostBar Intelligent Mail Data Logic Leitcode Pharmacode PZN ISBN Composite Aztec Mesas EAN.UCC EAN-14 Comp. RSS-14 Comp. RSS-14 St. Comp.
  47. Interleaved 2 of 5 Code 128 Code 128 M axicode

    M icro QR Code 39 Code 39 Code 128 Code 39
  48. How to pick a symbology? Does your application dictate a

    specific symbology? Continue listening to this presentation. Use the symbology dictated by your application. Yes No
  49. Reader Technology Really bright lamps ✍ Wands ɾ Laser ɑ

    Imaging
  50. Reader Technology Really bright lamps ✍ Wands ɾ Laser ɑ

    Imaging Photos: “ACI plate on an IANR hopper” by Feddacheenee on Wikimedia Commons. https://commons.wikimedia.org/wiki/File:ACI_plate.jpg “Petrified Forest, Flora, Summer 2010” Quinn Rossi on flickr, Licensed under CC BY 2.0. https://www.flickr.com/photos/theeskimo/4898894840/
  51. Reader Technology Really bright lamps ✍ Wands ɾ Laser ɑ

    Imaging Photo: “Barcodelesestift mit Leseeinheit” by I,Nightflyer on Wikimedia.de under CC BY-SA 3.0 https://de.wikipedia.org/wiki/Barcodeleseger%C3%A4t#/media/File:Barcodelesestift.jpg
  52. Reader Technology Really bright lamps ✍ Wands ɾ Laser ɑ

    Imaging
  53. Reader Technology Really bright lamps ✍ Wands ɾ Laser ɑ

    Imaging Photo: “Swap Paper for Mobile Boarding Passes“ by City of Mc Allen.
  54. None
  55. None
  56. Printing Technology

  57. Direct Parts Marking: Permanently marking a piece by punching, etching,

    laser, etc. “Printing” Technology
  58. “Printing” Technology Otolith Marking: Information stored in fish bones for

    fishery management Photo: randychiu on flickr, Licensed under CC BY 2.0. https://www.flickr.com/photos/randychiu/4817651586/
  59. Barcodes with Python

  60. Generating Barcodes >>> import barcode >>> bc = barcode.get_barcode('code39', 'PYBAY2018')

    >>> bc.save('mybc') 'mybc.svg' pip install python-barcode
  61. Generating Barcodes >>> import barcode >>> barcode.generate('code39', 'PYBAY2018', output='mybc') 'mybc.svg'

    pip install viivakoodi
  62. Generating Barcodes >>> from pubcode import Code128 >>> bc =

    Code128('PyBay2018', charset='B') >>> bc.bars '21121431312121214113112312112421214122321112312212322131122 >>> bc.image(height=50).save('mybarcode.png') MissingDependencyError: PIL module is required to use image pip install PubCode
  63. Generating Barcodes >>> from pubcode import Code128 >>> bc =

    Code128('PyBay2018', charset='B') >>> bc.bars '21121431312121214113112312112421214122321112312212322131122 >>> bc.image(height=50).save('mybarcode.png') pip install PubCode pillow
  64. Generating Barcodes import pdf417gen, this data = pdf417gen.encode(this.s) image =

    pdf417gen.render_image(data) image.save('mybarcode.png') pip install pdf417gen
  65. ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓

    ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ Generating Barcodes treepoem pyBarcode ↳ python-barcode ↳ viivakoodi ↳ steenzout.barcode ↳ reBarcode pyStrich (fork of huBarcode) qrcode PyQRCode segno pdf417gen code128 PubCode candybar License MIT MIT MIT MIT MIT MIT Apache BSD BSD BSD MIT LGPL MIT Apache UPC-A EAN ISBN PZN I. 2 of 5 Code 39 Code 93 Code 128 Data Matrix PDF417 QR Micro QR ★ 123 N/A 17 28 5 0 N/A 1,481 204 18 13 N/A 1 2 Latest release June 2018 July 2013* June 2018 Nov 2014 Jan2017 Oct 2017 July 2016 Mar 2018 June 2016 Feb 2018 May 2017 Jan 2015 Aug 2015 Aug 2016 Output 1: SVG 2: pillow 3: pypng []: optional 2 1, [2] 1, [2] 1, [2] 1, [2] 1, [2] 2 1, 3, [2] 1, 3 2 1, [2] 2 2 ✓ Aztec various formats, no external dep.
  66. Generating Barcodes Easy to install, support for common symbologies: python-barcode

    license: MIT, last updated: June 2018 https://pypi.org/project/python-barcode/ Most feature rich tool: Treepoem (uses BWIPP, requires Postscript) license: MIT, last updated: Aug 2018 https://pypi.org/project/treepoem/ Symbology specific libraries: Segno (QR, Micro QR), pdf417gen, libdmtx (Datamatrix), PubCode (Code 128)
  67. Reading Barcodes with Python

  68. Don’t read barcodes with Python Use purpose-built hardware whenever possible.

    These can optimize every step of the process going from light to data. Photos: “Symbol Motorola” by Lost Parcels on flickr, licensed under CC BY 2.0, https://www.flickr.com/photos/lostparcels/31046764866/. By HP APJ on flickr, licensed under CC BY-NC-ND 2.0, https://www.flickr.com/photos/personalagain/5444387315/
  69. Reading Barcodes with Python Pick your poison: zbar C/C++ library

    EAN-8/13, UPC-A/E, EAN-8, Interleaved 2 of 5, Code 39, Code 128, QR (not documented: Codabar, Code 93, PDF417) license: LGPL 2.0, last updated: Oct 2009 https://sourceforge.net/projects/zbar/ ZXing Java library (and C++ rewrite/port/fork) same as zbar, plus: Codabar, Code 93, ITF, Data Matrix, Aztec, PDF417, Maxicode, GS1 Databar (and various codes using these symbologies) license: MIT, last updated: Aug 2018 https://github.com/zxing/zxing (Java) https://github.com/glassechidna/zxing-cpp (C++) libdmtx C/C++ library, .NET fork, Datamatrix symbology only license: LGPL 2.0, last updated: unclear https://sourceforge.net/projects/libdmtx/, https://github.com/dmtx, https://sourceforge.net/projects/datamatrixnet/
  70. “The Original” Last updated 2009 Python 2 only Actively maintained

    Python 2 and Python 3 Actively maintained Advertises easy Windows installation
  71. There exists an sparsely maintained C++ rewrite/port/fork of ZXing ().

    This wraps that. https://github.com/glassechidna/zxing-cpp https://github.com/lubo/zxinglight - Setup.py downloads .jars - uses subprocess.Popen() Python 3 only
  72. ZXing example >>> import zxing >>> reader = zxing.BarCodeReader() >>>

    barcode = reader.decode("boardingpass.png") BarCode('420941079374889676090739719680', '420941079374889676090739719680', 'CODE_128', 'TEXT', [(37.5, 96.0), (805.0, 96.0)]) pip install zxing
  73. Reading for Ȑ : Commercial SDKs Python support advertised Scandit

    (C) https://www.scandit.com/products/barcode-scanner/barcod e-scanner-details/ Cognex ManateeWorks https://manateeworks.com/barcode-scanner-sdk Vintasoft (.NET) http://www.vintasoft.com/vsbarcode-dotnet-index.html Leadtools Barcode SDK (Java, C, .NET Core) https://www.leadtools.com/sdk/barcode DTKSoft Barcode SDK (Windows only) https://www.dtksoft.com/?p=barcodesdk Sension (C++) https://www.sensionweb.com/codemax-barcode-sdk Python support advertised Dynamsoft Barcode Reader https://www.dynamsoft.com/ https://pypi.org/project/dbr/ Matrox Imaging Library https://www.matrox.com/imaging/en/products/software/mil/ (Python wrapper available as “update”) NO endorsement is implied. Do yoUR own rESEarch!
  74. None
  75. None
  76. Barcodes in the Realworld

  77. Barcodes as real-world Browser Cookies

  78. Barcodes as Information Photo: “Heinz QR porn code too saucy

    for ketchup customer”, BBC, Jun https://www.bbc.com/news/technology-33200142
  79. Barcodes as Passwords (spoiler: don’t do it)

  80. Barcodes as Passwords (spoiler: don’t do it) >>> barcode =

    reader.decode("boardingpass.png").raw 'M1NEUBERT/JONAS EP8LD5W SFOSANUA 1900 034Y029D0071 15D>5180 W8034BUA 2A01123810700013 UA UA GV444737 N*30600 05 ^QO6IGQATNXzqrr57ZahTjZGbeLkuah2ZCvlKPeBBodwewJDZWTitJftePwRGUZs F7yCngxRxKj9x9MtIAbgLE21TueaF4GIT7QU'
  81. Barcodes as Passwords (spoiler: don’t do it)

  82. RTFM (part 1) Understand the hardware you are using A

    barcode reader with factory settings is like a Windows PC without a firewall
  83. RTFM (part 2) Understand the symbology you are using Example:

    With the Interleaved 2-of-5 symbology makes valid partial scans very likely, only use with fixed length content I AM A RED STICKER THAT BREAKS YOUR BARCODE “4321” “0987654321”
  84. RTFM (part 3) Understand the code you are using Example:

    Google Chrome “crashes” when scanning a product code into a textbox Here’s why: - Product code is a GS1 Datamatrix code - GS1 Datamatrix specifies “FNC1” character as separator - Code 128 maps “FNC1” to 0x64 - My barcode reader decides to communicate 0x64 as F8 key - In Chrome developer tools “F8” means: Stop code execution
  85. Tips for working with barcodes - “Outsource” the decoding if

    you can - Use a symbology that is right and safe for your data type - Configure your reader hardware - Limit the symbologies accepted to those you actually need - Disable reader configuration through barcodes - Validate input as if it was user input (because it is) - Validate that only the characters or numbers you expect are used - Check length of data (most symbologies are variable length) - Protect against Little Bobby Barcodes injection attacks - If your software hands out barcodes, treat them like browser cookies - Never include secrets in barcodes
  86. My last slide What’s the URL for these slides again?

    http://jonasneubert.com/talks/pybay2018.html Want to work with me? https://www.zymergen.com/careers/ Watching this on Youtube? The best ways to reach me are Twitter (@jonemo) and email (jn@jonasneubert.com)
  87. Books Roger C. Palmer: The Bar Code Book: A Comprehensive

    Guid To Reading, Printing, Specifying, Evaluating, And Using Bar Code and Other Machine-Readable Symbols, 5th ed., Trafford Pub. 2007 John Berry: The Secret Life of Bar Codes, Wirksworth Books, 2013 Gavin Weightman: Eureka: How Invention Happens, Yale University Press, 2017. Excerpt available at https://www.smithsonianmag.com/innovation/history-bar-code-180956704/ — I didn’t read the following but came across references to them during my research for this presentation: Ben Nelson: Punched Cards to Barcodes: A 200 Year Journey with Descriptions of over 260 Cods, Helmers Publishing, 1997. Stephen A. Brown: Revolution at the Checkout Counter: The Explosion of the Bar Code, Wertheim Publications in Industrial Relations, 1997. Bill Selmeier: Spreading the Barcode, Lulu.com, 2010. George J. Laurer: Engineering Was Fun, 3rd ed., Lulu.com, 2012.
  88. Articles Katie Mingle: Barcodes. 99 Percent Invisible. 2014. https://99percentinvisible.org/episode/barcodes/ Ernie

    Smith: Right Track, Wrong Station, Tedium. 2017. https://tedium.co/2017/09/05/kartrak-railroad-barcode-history/ Ernie Smith: Switching Labels, Tedium. 2017. https://tedium.co/2017/11/13/retail-theft-ticket-switching/ Ernie Smith: When Tech Hit Retail, Tedium. 2017. https://tedium.co/2017/07/20/point-of-sale-retail-history/ Larry Silverman: Barcodes, A Brief History, TrackAbout, 2015. https://corp.trackabout.com/barcodes-brief-history Brian Krebs: Why It’s Still A Bad Idea to Post or Trash Your Airline Boarding Pass, 2017. https://krebsonsecurity.com/2017/08/why-its-still-a-bad-idea-to-post-or-trash-your-airline-boarding-pass/ Jonas Neubert: Intro to Barcode Readers, 2017. http://jonemo.github.io/neubertify/2017/02/26/intro-to-barcode-readers/ Shaun Ewing: What’s contained in a boarding pass barcode, 2011. https://shaun.net/notes/whats-contained-in-a-boarding-pass-barcode/
  89. Videos & Talks Barcode in Black and White, Groupe Média

    TFO. https://www.youtube.com/watch?v=FXPSd5o0GeU (published to Youtube by OCR Canada Ltd.) “FX” Felix Lindner: Toying with Barcodes, DEFCON 16 (2008). https://www.youtube.com/watch?v=qT_gwl1drhc Karsten Nohl, Nemanja Nikodijevic: Where in the World Is Carmen San Diego, Becoming a Secret Travel Agent, 33c3 (2016). https://media.ccc.de/v/33c3-7964-where_in_the_world_is_carmen_sandiego Karina Ruzinov: Shipping secret messages through barcodes, PyCascades 2018. https://www.youtube.com/watch?v=01E5exVatGA
  90. Online Resources (besides Wikipedia) Russ Adams: Barcode 1. http://www.adams1.com/ Retail

    Identification History Museum. http://www.idhistory.com/ Egoditor UG: http://www.barcode-generator.org/ (scroll down past the generator for content)
  91. The Original Barcode Patent US2612994A: Classifying apparatus and method, 1949,

    Norman J Woodland, Silver Bernard (Drexel University) Otolith Marking (Barcodes in Fish Bones) Volk, Schroder, Grimm, Ackley: Use Of A Bar Code Symbology to Produce Multiple Thermally Induced Otolith Marks, Transactions of the American Fisheries Society, vol. 123, is. 5, pp. 811-816 (1994). Volk, Schroder, Grimm: Otolith thermal marking, Fisheries Research, vol. 43, is. 1-3, pp. 205-219 (1999). Helmut Schraets: Meilensteine der Briefcodierung (1998). https://www.briefmarken-kevelaer.de/media/files/Meilensteine-Neu.pdf USPS: Intelligent Mail Barcode 4-state Specification (2015). https://ribbs.usps.gov/intelligentmail_mailpieces/documents/tech_guides/USPSB3200IntelligentMailBarco de4State.pdf DNA Bardcoding https://en.wikipedia.org/wiki/DNA_barcoding
  92. Image Attributions Zebra by m. turan ercan from the Noun

    Project: https://thenounproject.com/term/zebra/67862/