Moving PTL to Python 3

Owned by Ravi Vasista (Deactivated)
Last updated: Jan 31, 2019
2 min read

  • Links that might be useful

    It is interesting to note that the tool Py2to3, always returns the same object that the function used in Python 2 did, even when it is unnecessary. A few examples of this are given below - 


    •   - dict(map(map_fun, cols, row))
       + dict(list(map(map_fun, cols, row)))

      • Python 2 returns a list when map is used and Python 3 returns a map object. The map object can be directly cast to a dictionary if needed.


    •   - data.iteritems():
       + data.items():

      • Python 2 returns a generator for dict.iteritems() so the tool converts this in Python 3 to dict.items()


    •   - data.items():
       + list(data.items()):

      • In this case however, dict.items() returns a list in Python 2 which is why the tool converted the generator returned by dict.items() explicitly to a list.


    •   - data.keys()
       + list(data.keys())

      • Similar to the previous case. Explicit conversion to a list in case of dict.keys()


    • base64bytes = base64.b64encode("**Random String**".encode('utf-8'))

      base64bytes.decode('utf-8')

      • This change is manual. Py2to3 does not take this into account. The b64encode function requires a byte-like object and returns one as well in Python 3, this was not the case for Python 2.


    •   - urllib2.Request(url, json.dumps(data))
       + urllib.request.Request(url, json.dumps(data))

      • urllib2 has changed to urllib and it's functions have been moved to sub packages.

         

    •   - httplib.HTTPConnection(self.host, timeout=25)
       + http.client.HTTPConnection(self.host, timeout=25)

      • httplib has changed to http and it's functions have been moved to sub packages.


    •   - filter(lambda x: x[0] == '**match**', filters):
       + [x for x in filters if x[0] == '**match**']:

      • Filter like map now returns a filter object. 


    • Exceptions in Python 3 no longer have a message attribute. This has been replaced with str(e). This change is also manual.


Specifically from a PTL stand point - 

  • The defusedxml pypi page says it's for Python 2.7 but it is available through pip.
  • The binaries have to have the .py extension, otherwise they will not be considered while running the tool for the workspace.
  • There are issues while converting pbs_as and the conversion fails. This has to be done manually. 
    RefactoringTool: Skipping optional fixer: buffer
    RefactoringTool: Skipping optional fixer: idioms
    RefactoringTool: Skipping optional fixer: set_literal
    RefactoringTool: Skipping optional fixer: ws_comma
    RefactoringTool: Can't parse pbs_dev/pbspro/test/fw/bin/pbs_as: ParseError: bad input: type=1, value='elif', context=('\n ', (341, 4))
    RefactoringTool: No files need to be modified.
    RefactoringTool: There was 1 error:
    RefactoringTool: Can't parse pbs_dev/pbspro/test/fw/bin/pbs_as: ParseError: bad input: type=1, value='elif', context=('\n ', (341, 4))



Link to Forum - http://community.pbspro.org/t/moving-to-python-3/1421



OSS Site Map

Project Documentation Main Page

Developer Guide Pages