Share
Embed
Zebras & Lasers (PyBay 2018)
Jonas Neubert
August 18, 2018
Transcript
92 slides
-
Zebras and Lasers: A crash course on barcodes with Python
Jonas Neubert 3rd Annual Regional Python Conference August 16 - 19, 2018 | San Francisco, CA -
From First Principles ʵ Barcodes Theory ż Barcodes with Python
Ʉ Barcodes in The Realworld -
Symbology - Geometric primitives (lines, circles, matrix, ...) - Arrangement
of primitives (single row, concentric, stacked, …) - Symbol table (pixels → bits) - Encoding (bits → information) - Checksums and Error Correction Code - What the numbers mean (part numbers, ZIP codes, ...) -
11 slots 3 bars & 3 gaps per character Maximum
Bar Size: 4 Slots 106 Different Characters 3 Character Sets for “Full-Ascii” Code 128: Full ASCII, variable length, continuous (11,3), three character sets -
character “P” (0x30) Code 128: Full ASCII, variable length, continuous
(11,3), three character sets -
character “H”(0x28) Code 128: Full ASCII, variable length, continuous (11,3),
three character sets -
character “P”(0x30) Code 128: Full ASCII, variable length, continuous (11,3),
three character sets -
“PHP” cHecksUm (0x43) Code 128: Full ASCII, variable length, continuous
(11,3), three character sets -
“PHP” stARt coDE (0x68) STOP Code ..(0x70) cHecksUm (0x43) Code
128: Full ASCII, variable length, continuous (11,3), three character sets -
“PHP” stARt coDE (0x68) STOP Code ..(0x70) cHecksUm (0x43) quiet
ZonEs Code 128: Full ASCII, variable length, continuous (11,3), three character sets -
Code 39: Numbers and uppercase letters (39 chars), discrete 3-of-9
2-width 12 slots 5 bars & 4 gaps per character Two distinct widths 3 out of 9 Bars or Gaps are 2x wide Variable Gap between characters 39 Different Characters: A-Z 0-9 *$/+% -
* Start code ? Just use “*” STOP CODE? Another
“*” cHecksUm optional * H P P Code 39: Numbers and uppercase letters (39 chars), discrete 3-of-9 2-width -
0 8 8 1 Start Guard StoP Guard cHecksUm optional
Interleaved 2-of-5: Numbers only, variable length, continuous 2-of-5 2-width -
Start GUard STOP guarD cHecksUm 9 4 1 4 3
3 0 0 8 - POSTNET: Numbers only, fixed length, clocked continuous 2-of-5, height-modulated USPS pre-2009 -
Zielcode: Numbers only, fixed length, continuous 2-of-5 1-width Deutsche Post
Source: Helmut Schraets: Meilensteine der Briefcodierung (1998). https://www.briefmarken-kevelaer.de/media/files/Meilensteine-Neu.pdf -
Intelligent Mail®: Numbers only, variable length, clocked 4-state, height-modulated US
Postal since 2009 00-300-123456-123456789-94143-3008-00 “Send First Class letter with no services (300) no OEL information (00) to UCSF Conference center (94143-3008-00) using mAILer ID 123456 and sequence number 123456789” -
Source: USPS: Intelligent Mail Barcode 4-state Specification (2015), Intelligent Mail®:
Numbers only, variable length, clocked 4-state, height-modulated US Postal since 2009 -
Rule of Thumb: Height = 0.15 * Width Anonymous 1980s
engineer pulling Number out of his Hat Bacon -
Bacon & Eggs Rule of Thumb: Height = 0.15 *
Width Anonymous 1980s engineer pulling Number out of his Hat -
Bacon & Egg with Toast Rule of Thumb: Height =
0.15 * Width Anonymous 1980s engineer pulling Number out of his Hat -
Code 49: Stacked linear codes, fixed length (49 chars) Full
Breakfast with Bacon & Eggs and Toast & Jam -
Code 16k: Stack of inverted Code 128 symbols, 77 chars
or 154 digits -
PDF417: “High capacity”, variable length, stack of continuous (17,4) codes
Full Breakfast with Bacon, Baked Beans and Eggs and Toast or an English Muffin as a side. -
Data Matrix: Arbitrary data, up to 1556 bytes/2335 chars/3116 digits
White in Top right means: Uses EC200 PyBay Left and bottom guards are solid Top and right guards are clocked -
Data Matrix: Arbitrary data, up to 1556 bytes/2335 chars/3116 digits
White in Top right means: Uses EC200 PyBay Left and bottom guards are solid Top and right guards are clocked -
Data Matrix: Arbitrary data, up to 1556 bytes/2335 chars/3116 digits
White in Top right means: Uses EC200 PyBay Left and bottom guards are solid Top and right guards are clocked P y B P y a B a REED-SOLOMON ERROR CORRECTION PADDING -
Semacode: Code for storing URLs in Datamatrix Source: Bernd Hopfengärtner:
Hello World! (2007) http://hello.w0r1d.net/ -
QR: up to 7089 digits or 4296 ASCII characters or
1817 Kanji characters -
QR: up to 7089 digits or 4296 ASCII characters or
1817 Kanji characters Source: Xinhua (2017) http://www.xinhuanet.com/photo/2017-09/14/c_1121662356_2.htm -
Aztec: up to 3832 digits or 3067 characters Maxicode: exactly
596 bits, hexagonal elements Grid Matrix: optimized for encoding Chinese characters Code One: First non-proprietary 2D symbology (1992) Micro QR: Variation of QR Code, numeric, up to 35 digits, single alignment mark ← HIBC Datamatrix: Variation of Datamatrix, up to 36 characters, used in medical devices ← -
UPC/EAN: Numbers only, fixed length of 12 or 13 digits,
continuous (7,2) 4-width Source: Businessweek, April 7, 1973. Available on the website of the ID History Museum at http://idhistory.com/ibm/Busweek1973001.pdf -
1D Interleaved 2-of-5 Code 39 Code 128 :CueCat Bullseye UPC/EAN
Stacked Code 49 Code 16k PDF417 Matrix Datamatrix Aztec Maxicode Grid Matrix Code One QR Mini QR POSTNET RM4SCC PostBar Intelligent Mail -
1D Interleaved 2-of-5 Code 39 Code 128 :CueCat Codabar Code
11 Code 93 93i Channel Code Bullseye UPC-A, UPC-E EAN-13, EAN-8 GS1-128 GS1 Databar (RSS) Plessey Code Stacked Code 49 Code 16k PDF417 MicroPDF417 Codablock SuperCode UltraCode GS1 Databar St. Matrix Datamatrix Aztec Maxicode Grid Matrix Code One QR Mini QR Vericode MiniCode GoCode POSTNET RM4SCC PostBar Intelligent Mail Data Logic Leitcode Pharmacode PZN ISBN Composite Aztec Mesas EAN.UCC EAN-14 Comp. RSS-14 Comp. RSS-14 St. Comp. -
Interleaved 2 of 5 Code 128 Code 128 M axicode
M icro QR Code 39 Code 39 Code 128 Code 39 -
How to pick a symbology? Does your application dictate a
specific symbology? Continue listening to this presentation. Use the symbology dictated by your application. Yes No -
Reader Technology Really bright lamps ✍ Wands ɾ Laser ɑ
Imaging Photos: “ACI plate on an IANR hopper” by Feddacheenee on Wikimedia Commons. https://commons.wikimedia.org/wiki/File:ACI_plate.jpg “Petrified Forest, Flora, Summer 2010” Quinn Rossi on flickr, Licensed under CC BY 2.0. https://www.flickr.com/photos/theeskimo/4898894840/ -
Reader Technology Really bright lamps ✍ Wands ɾ Laser ɑ
Imaging Photo: “Barcodelesestift mit Leseeinheit” by I,Nightflyer on Wikimedia.de under CC BY-SA 3.0 https://de.wikipedia.org/wiki/Barcodeleseger%C3%A4t#/media/File:Barcodelesestift.jpg -
Reader Technology Really bright lamps ✍ Wands ɾ Laser ɑ
Imaging Photo: “Swap Paper for Mobile Boarding Passes“ by City of Mc Allen. -
Direct Parts Marking: Permanently marking a piece by punching, etching,
laser, etc. “Printing” Technology -
“Printing” Technology Otolith Marking: Information stored in fish bones for
fishery management Photo: randychiu on flickr, Licensed under CC BY 2.0. https://www.flickr.com/photos/randychiu/4817651586/ -
Generating Barcodes >>> import barcode >>> bc = barcode.get_barcode('code39', 'PYBAY2018')
>>> bc.save('mybc') 'mybc.svg' pip install python-barcode -
Generating Barcodes >>> from pubcode import Code128 >>> bc =
Code128('PyBay2018', charset='B') >>> bc.bars '21121431312121214113112312112421214122321112312212322131122 >>> bc.image(height=50).save('mybarcode.png') MissingDependencyError: PIL module is required to use image pip install PubCode -
Generating Barcodes >>> from pubcode import Code128 >>> bc =
Code128('PyBay2018', charset='B') >>> bc.bars '21121431312121214113112312112421214122321112312212322131122 >>> bc.image(height=50).save('mybarcode.png') pip install PubCode pillow -
Generating Barcodes import pdf417gen, this data = pdf417gen.encode(this.s) image =
pdf417gen.render_image(data) image.save('mybarcode.png') pip install pdf417gen -
✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓
✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ Generating Barcodes treepoem pyBarcode ↳ python-barcode ↳ viivakoodi ↳ steenzout.barcode ↳ reBarcode pyStrich (fork of huBarcode) qrcode PyQRCode segno pdf417gen code128 PubCode candybar License MIT MIT MIT MIT MIT MIT Apache BSD BSD BSD MIT LGPL MIT Apache UPC-A EAN ISBN PZN I. 2 of 5 Code 39 Code 93 Code 128 Data Matrix PDF417 QR Micro QR ★ 123 N/A 17 28 5 0 N/A 1,481 204 18 13 N/A 1 2 Latest release June 2018 July 2013* June 2018 Nov 2014 Jan2017 Oct 2017 July 2016 Mar 2018 June 2016 Feb 2018 May 2017 Jan 2015 Aug 2015 Aug 2016 Output 1: SVG 2: pillow 3: pypng []: optional 2 1, [2] 1, [2] 1, [2] 1, [2] 1, [2] 2 1, 3, [2] 1, 3 2 1, [2] 2 2 ✓ Aztec various formats, no external dep. -
Generating Barcodes Easy to install, support for common symbologies: python-barcode
license: MIT, last updated: June 2018 https://pypi.org/project/python-barcode/ Most feature rich tool: Treepoem (uses BWIPP, requires Postscript) license: MIT, last updated: Aug 2018 https://pypi.org/project/treepoem/ Symbology specific libraries: Segno (QR, Micro QR), pdf417gen, libdmtx (Datamatrix), PubCode (Code 128) -
Don’t read barcodes with Python Use purpose-built hardware whenever possible.
These can optimize every step of the process going from light to data. Photos: “Symbol Motorola” by Lost Parcels on flickr, licensed under CC BY 2.0, https://www.flickr.com/photos/lostparcels/31046764866/. By HP APJ on flickr, licensed under CC BY-NC-ND 2.0, https://www.flickr.com/photos/personalagain/5444387315/ -
Reading Barcodes with Python Pick your poison: zbar C/C++ library
EAN-8/13, UPC-A/E, EAN-8, Interleaved 2 of 5, Code 39, Code 128, QR (not documented: Codabar, Code 93, PDF417) license: LGPL 2.0, last updated: Oct 2009 https://sourceforge.net/projects/zbar/ ZXing Java library (and C++ rewrite/port/fork) same as zbar, plus: Codabar, Code 93, ITF, Data Matrix, Aztec, PDF417, Maxicode, GS1 Databar (and various codes using these symbologies) license: MIT, last updated: Aug 2018 https://github.com/zxing/zxing (Java) https://github.com/glassechidna/zxing-cpp (C++) libdmtx C/C++ library, .NET fork, Datamatrix symbology only license: LGPL 2.0, last updated: unclear https://sourceforge.net/projects/libdmtx/, https://github.com/dmtx, https://sourceforge.net/projects/datamatrixnet/ -
“The Original” Last updated 2009 Python 2 only Actively maintained
Python 2 and Python 3 Actively maintained Advertises easy Windows installation -
There exists an sparsely maintained C++ rewrite/port/fork of ZXing ().
This wraps that. https://github.com/glassechidna/zxing-cpp https://github.com/lubo/zxinglight - Setup.py downloads .jars - uses subprocess.Popen() Python 3 only -
ZXing example >>> import zxing >>> reader = zxing.BarCodeReader() >>>
barcode = reader.decode("boardingpass.png") BarCode('420941079374889676090739719680', '420941079374889676090739719680', 'CODE_128', 'TEXT', [(37.5, 96.0), (805.0, 96.0)]) pip install zxing -
Reading for Ȑ : Commercial SDKs Python support advertised Scandit
(C) https://www.scandit.com/products/barcode-scanner/barcod e-scanner-details/ Cognex ManateeWorks https://manateeworks.com/barcode-scanner-sdk Vintasoft (.NET) http://www.vintasoft.com/vsbarcode-dotnet-index.html Leadtools Barcode SDK (Java, C, .NET Core) https://www.leadtools.com/sdk/barcode DTKSoft Barcode SDK (Windows only) https://www.dtksoft.com/?p=barcodesdk Sension (C++) https://www.sensionweb.com/codemax-barcode-sdk Python support advertised Dynamsoft Barcode Reader https://www.dynamsoft.com/ https://pypi.org/project/dbr/ Matrox Imaging Library https://www.matrox.com/imaging/en/products/software/mil/ (Python wrapper available as “update”) NO endorsement is implied. Do yoUR own rESEarch! -
Barcodes as Information Photo: “Heinz QR porn code too saucy
for ketchup customer”, BBC, Jun https://www.bbc.com/news/technology-33200142 -
Barcodes as Passwords (spoiler: don’t do it) >>> barcode =
reader.decode("boardingpass.png").raw 'M1NEUBERT/JONAS EP8LD5W SFOSANUA 1900 034Y029D0071 15D>5180 W8034BUA 2A01123810700013 UA UA GV444737 N*30600 05 ^QO6IGQATNXzqrr57ZahTjZGbeLkuah2ZCvlKPeBBodwewJDZWTitJftePwRGUZs F7yCngxRxKj9x9MtIAbgLE21TueaF4GIT7QU' -
RTFM (part 1) Understand the hardware you are using A
barcode reader with factory settings is like a Windows PC without a firewall -
RTFM (part 2) Understand the symbology you are using Example:
With the Interleaved 2-of-5 symbology makes valid partial scans very likely, only use with fixed length content I AM A RED STICKER THAT BREAKS YOUR BARCODE “4321” “0987654321” -
RTFM (part 3) Understand the code you are using Example:
Google Chrome “crashes” when scanning a product code into a textbox Here’s why: - Product code is a GS1 Datamatrix code - GS1 Datamatrix specifies “FNC1” character as separator - Code 128 maps “FNC1” to 0x64 - My barcode reader decides to communicate 0x64 as F8 key - In Chrome developer tools “F8” means: Stop code execution -
Tips for working with barcodes - “Outsource” the decoding if
you can - Use a symbology that is right and safe for your data type - Configure your reader hardware - Limit the symbologies accepted to those you actually need - Disable reader configuration through barcodes - Validate input as if it was user input (because it is) - Validate that only the characters or numbers you expect are used - Check length of data (most symbologies are variable length) - Protect against Little Bobby Barcodes injection attacks - If your software hands out barcodes, treat them like browser cookies - Never include secrets in barcodes -
My last slide What’s the URL for these slides again?
http://jonasneubert.com/talks/pybay2018.html Want to work with me? https://www.zymergen.com/careers/ Watching this on Youtube? The best ways to reach me are Twitter (@jonemo) and email ([email protected]) -
Books Roger C. Palmer: The Bar Code Book: A Comprehensive
Guid To Reading, Printing, Specifying, Evaluating, And Using Bar Code and Other Machine-Readable Symbols, 5th ed., Trafford Pub. 2007 John Berry: The Secret Life of Bar Codes, Wirksworth Books, 2013 Gavin Weightman: Eureka: How Invention Happens, Yale University Press, 2017. Excerpt available at https://www.smithsonianmag.com/innovation/history-bar-code-180956704/ — I didn’t read the following but came across references to them during my research for this presentation: Ben Nelson: Punched Cards to Barcodes: A 200 Year Journey with Descriptions of over 260 Cods, Helmers Publishing, 1997. Stephen A. Brown: Revolution at the Checkout Counter: The Explosion of the Bar Code, Wertheim Publications in Industrial Relations, 1997. Bill Selmeier: Spreading the Barcode, Lulu.com, 2010. George J. Laurer: Engineering Was Fun, 3rd ed., Lulu.com, 2012. -
Articles Katie Mingle: Barcodes. 99 Percent Invisible. 2014. https://99percentinvisible.org/episode/barcodes/ Ernie
Smith: Right Track, Wrong Station, Tedium. 2017. https://tedium.co/2017/09/05/kartrak-railroad-barcode-history/ Ernie Smith: Switching Labels, Tedium. 2017. https://tedium.co/2017/11/13/retail-theft-ticket-switching/ Ernie Smith: When Tech Hit Retail, Tedium. 2017. https://tedium.co/2017/07/20/point-of-sale-retail-history/ Larry Silverman: Barcodes, A Brief History, TrackAbout, 2015. https://corp.trackabout.com/barcodes-brief-history Brian Krebs: Why It’s Still A Bad Idea to Post or Trash Your Airline Boarding Pass, 2017. https://krebsonsecurity.com/2017/08/why-its-still-a-bad-idea-to-post-or-trash-your-airline-boarding-pass/ Jonas Neubert: Intro to Barcode Readers, 2017. http://jonemo.github.io/neubertify/2017/02/26/intro-to-barcode-readers/ Shaun Ewing: What’s contained in a boarding pass barcode, 2011. https://shaun.net/notes/whats-contained-in-a-boarding-pass-barcode/ -
Videos & Talks Barcode in Black and White, Groupe Média
TFO. https://www.youtube.com/watch?v=FXPSd5o0GeU (published to Youtube by OCR Canada Ltd.) “FX” Felix Lindner: Toying with Barcodes, DEFCON 16 (2008). https://www.youtube.com/watch?v=qT_gwl1drhc Karsten Nohl, Nemanja Nikodijevic: Where in the World Is Carmen San Diego, Becoming a Secret Travel Agent, 33c3 (2016). https://media.ccc.de/v/33c3-7964-where_in_the_world_is_carmen_sandiego Karina Ruzinov: Shipping secret messages through barcodes, PyCascades 2018. https://www.youtube.com/watch?v=01E5exVatGA -
Online Resources (besides Wikipedia) Russ Adams: Barcode 1. http://www.adams1.com/ Retail
Identification History Museum. http://www.idhistory.com/ Egoditor UG: http://www.barcode-generator.org/ (scroll down past the generator for content) -
The Original Barcode Patent US2612994A: Classifying apparatus and method, 1949,
Norman J Woodland, Silver Bernard (Drexel University) Otolith Marking (Barcodes in Fish Bones) Volk, Schroder, Grimm, Ackley: Use Of A Bar Code Symbology to Produce Multiple Thermally Induced Otolith Marks, Transactions of the American Fisheries Society, vol. 123, is. 5, pp. 811-816 (1994). Volk, Schroder, Grimm: Otolith thermal marking, Fisheries Research, vol. 43, is. 1-3, pp. 205-219 (1999). Helmut Schraets: Meilensteine der Briefcodierung (1998). https://www.briefmarken-kevelaer.de/media/files/Meilensteine-Neu.pdf USPS: Intelligent Mail Barcode 4-state Specification (2015). https://ribbs.usps.gov/intelligentmail_mailpieces/documents/tech_guides/USPSB3200IntelligentMailBarco de4State.pdf DNA Bardcoding https://en.wikipedia.org/wiki/DNA_barcoding -
Image Attributions Zebra by m. turan ercan from the Noun
Project: https://thenounproject.com/term/zebra/67862/