Pragmatic Unicode, or, How do I stop the pain?

Channel:Next Day Video · Length:36:20 · Published:2012-03-16 · Views:80510

Watch on YouTube

Summary

Ned Batchelder's PyCon 2012 talk gives a quick overview of what Unicode is, then focuses on what a Python programmer needs to handle text correctly. It presents the 'Unicode sandwich' strategy: decode bytes at input, work with Unicode internally, encode at output.

Why HN recommends it

Commenters point to it as the clearest explanation of Python's bytes versus Unicode model and the Unicode sandwich, which finally makes the confusion click.

After A TON OF PAIN I decided to put the time in to watch the talk — HN, 2019

Python, Ruby & scripting languages · Graphic design & typography

Subtitles (excerpt)

final talk is by Ned belda pragmatic Unicode or how to stop the pain hello yeah hi everyone uh my name is Ned bachelder I've been working with python for well over 10 years um I'm going to talk pretty fast during this talk partly because I have a lot to say and partly because I grew up in New York City um I've been working with python for about 10 years which means at least a dozen times I've made all the same mistakes supporting Unicode that you probably have that have caused you pain that have brought you to this room to try and find a way to stop the pain so let's talk about how that happened what happened in the past was you wrote a program and you were really happy uh cuz it worked you had this great program um and then all of a sudden something weird happened a strange character appeared in your program and errors happened and you were sad okay that that is the past that's not going to happen anymore 25 minutes from now you will know everything you need to know to not have that happen but the way you're feeling right now is you're annoyed you're angry and and you're fundamentally uninterested in Unicode right so I'm going to try to keep this really straightforward and just tell you what

On Hacker News

You have to invest some time in understanding how strings (byte strings and unicode) work in Python. I put this off for way too long. After A TON OF PAIN I decided to put the time in to watch the talk "Pragmatic Unicode, or, How do I stop the pain?"[0] by Ned Batchelder and it all just clicked. Now, I make unicode sandwiches like a boss. [0] [VIDEO] — jteppinette, 2019-05
From a quick skimming through the docs[0], Perl 5.8's Unicode support sounds a lot like Python 2's except that the default encoding is latin1 instead of ASCII - i.e. unlike the Python 3 way of explicitly decoding binary data into Unicode text at the point where it's read (where presumably the encoding of the data is known), it will defer the decoding until the data is used (at some completely unr… — ptx, 2015-01
I don't know the details (because I don't understand unicode) but I am not sure what you said is possible. But to me ASCII seems like a hindrance. ``` fmt.Println("Hello, 世界") ``` This line of go lang looks beautiful. I bet they never have to worry about bytes vs Unicode. Here is python... [VIDEO] I'll add one more assertion to the list. People are born and people die. It is not fair for us to im… — thanksgiving, 2017-11
Python 3 is useful for a beginner to start with because they don't have to learn the deprecated cruft that Python 2 preserves for compatibility. If they just learn the Python 3 way of doing things, it generally works in both 2 and 3. In his explanation of classes he has several paragraphs about old-style vs. new-style classes, ending it with: "Just completely ignore the idea of old style versus n… — ptx, 2015-03

Related videos