44
55Python3.6 on XS8 does not have an all-encompassing default UTF-8 mode for I/O.
66
7- Newer Python versions have an UTF-8 mode that they even enable by default.
8- Python3.6 only enabled UTF-8 for I/O when an UTF-8 locale is used.
7+ Newer Python versions have a UTF-8 mode that they even enable by default.
8+ Python3.6 only enabled UTF-8 for I/O when a UTF-8 locale is used.
99See below for more background info on the UTF-8 mode.
1010
1111For situations where UTF-8 enabled, we have to specify UTF-8 explicitly.
1212
13- Such sitation happens when LANG or LC_ * variables are not set for UTF-8.
14- XAPI plugins like auto-cert-kit find themself in this situation.
13+ Such situation happens when LANG or LC_ * variables are not set for UTF-8.
14+ XAPI plugins like auto-cert-kit are in this situation.
1515
1616Example:
1717For reading UTF-8 files like the ` pciids ` file, add ` encoding="utf-8" ` .
18- This applies especailly to ` open() ` and ` Popen() ` when files my contain UTF-8.
18+ This applies especially to ` open() ` and ` Popen() ` when files my contain UTF-8.
1919
2020This also applies when en/decoding to/form ` urllib ` which uses bytes.
2121` urllib ` has to use bytes as HTTP data can of course also be binary, e.g. compressed.
@@ -159,25 +159,26 @@ tests/test_pci.py line 96 in TestPCIIds.test_videoclass_by_mock_calls()
159159tests/ test_pci.py line 110 in TestPCIIds.mock_lspci_using_open_testfile()
160160```
161161
162- Of course, `xcp/ net/ ifrename` won' t be affected but it would be good to fix the
162+ Of course, `xcp/ net/ ifrename` won' t be affected, but it would be good to fix the
163163warning for them as well in an intelligent way. See the proposal for that below.
164164
165165There are a couple of possibilities and Python because 2.7 does not support the
166166arguments we need to pass to ensure that all users of open () will work, we need
167167to make passing the arguments conditional on Python >= 3 .
168168
169- 1 . Overriding `open ()` , while technically working would not only affect xcp.python but the entire program:
169+ 1 . Overriding `open ()` .
170+ While technically working, it would affect the entire interpreter:
170171
171172 ```py
172173 if sys.version_info >= (3 , 0 ):
173174 original_open = __builtins__[" open" ]
174- def uopen (* args, ** kwargs):
175+ def utf8_open (* args, ** kwargs):
175176 if " b" not in (args[1 ] \
176177 if len (args) >= 2 else kwargs.get(" mode" , " " )):
177178 kwargs.setdefault(" encoding" , " UTF-8" )
178179 kwargs.setdefault(" errors" , " replace" )
179180 return original_open(* args, ** kwargs)
180- __builtins__[" open" ] = uopen
181+ __builtins__[" open" ] = utf8_open
181182 ```
182183
1831842 . This is sufficient but is not very nice:
@@ -188,7 +189,7 @@ to make passing the arguments conditional on Python >= 3.
188189 open_utf8args = {" encoding" : " utf-8" , " errors" : " replace" }
189190 else :
190191 open_utf8args = {}
191- # xcp/{cmd,pci,environ?,logger?}.py tests/test_{pci,biodevname ?,...?}.py
192+ # xcp/{cmd,pci,environ?,logger?}.py tests/test_{pci,biosdevname ?,...?}.py
192193 + from .utf8mode import open_utf8args
193194 ...
194195 - open (filename)
@@ -208,7 +209,7 @@ to make passing the arguments conditional on Python >= 3.
208209 return open (* args, encoding = " utf-8" , errors = " replace" , ** kwargs)
209210 else :
210211 utf8open = open
211- # xcp/{cmd,pci,environ?,logger?}.py tests/test_{pci,biodevname ?,...?}.py
212+ # xcp/{cmd,pci,environ?,logger?}.py tests/test_{pci,biosdevname ?,...?}.py
212213 + from .utf8mode import utf8open
213214 ...
214215 - open (filename)
0 commit comments