upgrading_buzzad_to_python_3.pdf

By Mir

Transcript

1. Upgrading Buzzad to Python 3
2. Python2 vs. Python3 1. Division ex) 5 / 2; python 2 -> 2; python 3 -> 2.5 (float) 5 / 2.0; python 2 -> 2.5; python 3 -> 2.5 5 // 2; python 2 -> 2; python 3 -> 2 5 // 2.0; python 2 -> 2.0; python 3 -> 2.0 2. Text versus binary data python2: str => can contain both text(decoded) and byte(encoded) data python3: str(python2; unicode) type can contain only text data with encode, format, isdecimal, isnumeric method; bytes type can contain only byte data with decode method ex) python2: str(b'3') == b'3' (len = 1) python3: str(b'3') == “b'3'” (len = 4)
3. Text vs. Byte Encoeded vs. Decoded (python 2) >>> a = u'\u00c3' >>> a u'\xc3' # decoded data >>> a = '\u00c3' >>> a '\\u00c3' # encoded data
4. Text vs. Byte Encoeded vs. Decoded (python 3) >>> a = '\u00c3' >>> a 'Ã' # decoded data >>> a = b'\u00c3' >>> a b'\\u00c3' # encoded data
5. Text vs. Byte
6. xrange, raising exceptions • xrange(python2) == range(python3) • Raising exceptions # Python 2.7.6 >>> raise IOError, "file error" --------------------------------------------------------------------------- IOError Traceback (most recent call last) ... IOError: file error # Python 3.6.5 >>> raise IOError, "file error" File "<ipython-input-10-25f049caebb0>", line 1 raise IOError, "file error" ^ SyntaxError: invalid syntax
7. next() and .next() # Python 2.7.6 >>> my_generator = (letter for letter in 'abcdefg') >>> next(my_generator) >>> my_generator.next() ‘b’ # Python 3.6.5 >>> my_generator = (letter for letter in 'abcdefg') >>> next(my_generator) ‘a’ >>> my_generator.next() … AttributeError: 'generator' object has no attribute 'next'
8. For-loop variables and the global namespace leak # Python 2.7.6 >>> i = 1 >>> print ‘before: i=’, i before: i = 1 >>> ‘comprehension: ‘, [i for i in range(5)] comprehension: [0, 1, 2, 3, 4] >>> print ‘after: i = ‘ i after: i = 4 # Python 3.6.5 >>> i = 1 >>> print(‘before: i =’, i) before: i = 1 >>> print(‘comprehension:’, [i for i in range(5)]) comprehension: [0, 1, 2, 3, 4] >>> print(‘after: i=’, i) after i = 1
9. Comparing unorderable types • In python3, TypeError is raised as warning if trying to compare unorderable types # Python 2.7.6 >>> print "[1, 2] > 'foo' = ", [1, 2] > 'foo' [1, 2] > 'foo' = False >>> print "(1, 2) > 'foo' = ", (1, 2) > 'foo' (1, 2) > 'foo' = True >>> print "[1, 2] > (1, 2) = ", [1, 2] > (1, 2) [1, 2] > (1, 2) = False # Python 3.6.5 >>> print("[1, 2] > 'foo' = ", [1, 2] > 'foo') … TypeError: unorderable types: list() > str()
10. Iterable objects and lists • below methods return iterable objects in python3 ◦ zip() ◦ map() ◦ filter() ◦ dictionary’s .keys() method ◦ dictionary’s .values() method ◦ dictionary’s .items() method
11. Banker’s rounding • https://en.wikipedia.org/wiki/Rounding#Round_half_to_even # Python 2.7.6 >>> round(15.5) 16.0 >>> round(16.5) 17.0 # Python 3.6.5 >>> round(15.5) 16.0 >>> round(16.5) 16.0
12. etc... apply asserts basestring buffer dict except exec execfile exitfunc filter funcattrs future future getcwdu has_key idioms import imports imports2 input intern isinstance itertools itertools_imports long map metaclass methodattrs ne next nonzero numliterals operator paren print raise raw_input reduce reload renames repr set_literal standarderror sys_exc throw ...
13. Overview 1. Upgrade Django version from 1.5 to 1.8. Done. 2. Upgrade other libraries to latest Python 2.7 versions. Done. 3. Run 2 to 3 migrations tools 4. Run Buzzad on Python 3
14. Run 2 to 3 migrations tools • Can I use Python 3? ◦ 4 packages aren’t supported. (excluded `futures`) ◦ Incf-countryutils We use this lib for finding region of each country. We can replace this lib with `pycountry_convert` $ pip install caniusepython3 $ caniusepython3 -r requirements.txt … Finding and checking dependencies ... You need 3 projects to transition to Python 3. Of those 3 projects, 3 have no direct dependencies blocking their transition: incf-countryutils pyslack pyvimeo ratelim
15. `Incf.countryutils` to `pycountry_convert` from incf.countryutils import transformations … region = \ transformations.cca_to_ctn(country_alpha_2) api_endpoint = self.API_ENDPOINT_ASIA \ if region == 'Asia' else self.API_ENDPOINT_US import pycountry_convert … region = \ pycountry_convert.country_alpha2_to_continent_co de(country_alpha_2) api_endpoint = self.API_ENDPOINT_ASIA if region == 'AS' else self.API_ENDPOINT_US
16. Run 2 to 3 migrations tools • Can I use Python 3? ◦ pyslack We can replace this lib with `slackclient` $ pip install caniusepython3 $ caniusepython3 -r requirements.txt … Finding and checking dependencies ... You need 3 projects to transition to Python 3. Of those 3 projects, 3 have no direct dependencies blocking their transition: pyslack pyvimeo ratelim
17. `pyslack` to `slackclient` import slack … slack.chat.post_message( channel, text, username='[{}] {}'.format( settings.SERVER_ENV.upper(), username, ), ) from slackclient import SlackClient … slack_token = settings.SLACK_API_KEY sc = SlackClient(slack_token) sc.api_call( "chat.postMessage", channel=channel, text=text, user='[{}] {}'.format( settings.SERVER_ENV.upper(), username, ), )
18. Run 2 to 3 migrations tools • Can I use Python 3? ◦ pyvimeo Upgrade 0.3.9 to 1.0.2 (pyvimeo not official library of python3, but we can use it ◦ latelim Same with pyvimeo (in doc, “Works in Py2 and Py3.”) => https://github.com/vimeo/vimeo.py/pull/47 $ pip install caniusepython3 $ caniusepython3 -r requirements.txt … Finding and checking dependencies ... You need 3 projects to transition to Python 3. Of those 3 projects, 3 have no direct dependencies blocking their transition: pyvimeo ratelim
19. conclusion • Don’t believe everything caniusepython3 says. • Man-up. Install Python 3 and see what breaks.
20. Run Buzzad on Python 3 • Translate python 2 code to python 3 ◦ 2to3 Automated code translation tool • First run! And fix it! => follow Fabric’s doc, we have to use fabric3 $ fab run … File "/Users/mireulim/.virtualenvs/adserver-python3/bin/fab", line 11, in <module> ... ImportError: cannot import name 'isMappingType'
21. Run Buzzad on Python 3 => run 2to3!! $ fab run … NameError: name 'xrange' is not defined $ 2to3 -w lockjoy/strings_* $ fab run … error, error, error…..
22. Run Buzzad on Python 3 => success!! $ fab run [localhost] local: python manage.py runserver 0.0.0.0:9001 2018-06-19 09:28:32,003 [14823] [INFO] Found credentials in environment variables. 2018-06-19 09:28:32,131 [14823] [INFO] Raven is not configured (logging is disabled). Please see the documentation for more information. 2018-06-19 09:28:33,354 [14833] [INFO] Found credentials in environment variables. 2018-06-19 09:28:33,474 [14833] [INFO] Raven is not configured (logging is disabled). Please see the documentation for more information. Performing system checks... System check identified no issues (0 silenced). June 19, 2018 - 09:28:35 Django version 1.10.8, using settings 'lockjoy.settings' Starting development server at http://0.0.0.0:9001/ Quit the server with CONTROL-C.
23. One more thing • Run 2to3 only files occurs error! • Pycurl error in macos => https://cscheng.info/2018/01/26/installing-pycurl-on-macos-high-sierra.html • ‘DynamicClassAttribute’ object don’t have ‘value’ attribute in python3 • from sets import Set ⇒ set • string objects don’t have decode() method in python3 • check all changes files generated by 2to3 with => https://docs.python.org/3/howto/pyporting.html#pyporting-howto • Import current folder’s files: from strings_jp import * => from .strings_jp import *