Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to
your earbuds. This is episode 153, recorded October 16, 2019. I'm Brian Okken. And I'm
Michael Kennedy. This week's episode is sponsored by DigitalOcean. We'll talk more about them later.
But first, Michael, could you extend my knowledge a bit? Yeah, by like extending
the entire Python ecosystem, maybe? Yeah. Yeah. So there's actually a cool real Python
article called Building a Python C Extension Module. So Brian, you know how to write C code,
right? Yes. Or at least that's the theory. I used to know how. Yeah. I have this really awesome
former self of mine that was super good at C++. Yeah. I kind of remember that person that I was.
I used to be able to write a lot of C. That was my main job is to write C and 3D stuff and OpenGL and
things like that, right? So it's definitely the main way to extend Python these days. And there's
other options like there's some cool Rust options and whatnot. But, you know, primarily people know
C. It runs everywhere, has like light runtime requirements. You're already running CPython
probably, so you already have those requirements met, right? So extending your code with some kind
of C extension gives you a couple of options. One is clearly performance. I love to talk about
Python performance because one, it always surprises me. And two, like people are usually wrong about it.
They say Python is slow. Like I was just reading something on Quora about why, like compare C#
to Python. And somebody says, well, you can't even compare them. C# is 50 times faster.
Well, that's true for certain operations, unless maybe part of that it's done in C and then probably
Python is faster because like now it's down in like NumPy and doing it in C, which is actually
faster, right? There's just, it gets really interesting. So one reason you might care about
writing a C module is just for performance. And I think that's what most people think of,
but there's also like low level operating system APIs or other C APIs, like some library you can get,
you might want to use that happens to be written in C and doesn't have a Python way to talk to it,
right? Yeah. There's lots of stuff with DLLs that are available with C header files,
but you don't have a Python binding. Exactly. And I bet you have a lot of experience with that,
with all of your devices and stuff like that. Yep. Yeah. Okay. So those are the two main reasons I can
think of writing C extensions. I mean, obviously throw some Cython at it if it's a performance thing
to give it a try, but there's a cool tutorial on real Python and it talks about how you can,
you know, like things you'll be able to do is like import C functions within Python,
pass arguments from Python to C, raise correct exceptions in your C code. So they surface bubble
back into your Python code as a proper, like a value error type exception or something like that,
all sorts of cool things and even how to test and distribute it. So let me just sort of talk
through the process and then people really care. They can go read the big long article, right? So
if you want to basically get access to some C functionality, or if you want to just like write
your implementation and see for some degree, first thing you got to do is go and figure out, let's
suppose you want to call like a, some C function, right? So the article uses F puts, which puts a string
into a file pointer, right? Like basically writes a string to a file in C. So you have to write a
function, which is pretty interesting because it returns, you have to start talking in the CPython
language, not Python, right? So everything that gets passed around is a Python object pointer or the
return value is like a py object pointer, right? So you pass these things around and first of all,
you declare like whatever inbound arguments you're really expecting and you get basically passed a
single pointer that is the arguments to your function, but that's really a tuple. So there's
a pyarg parse tuple, give you the arguments, a format thing, and then you give it the address of the
pointers. You pass them by reference basically. And then you just do your CPython code. In this case,
the function that they're wrapping, F puts, returns the number of bytes copied when it does that.
And so this function wants to return the bytes copied, but you can't just return an integer or a
long. No, no, no, because everything in Python is a py object at the C level, py object star,
even numbers. So you have to convert from a long to a py long from long, which is a function that you
get from the Python.hc header file. Okay.
It's actually pretty simple. There's like some weird non-obvious like structure at the beginning
of the function. So it can be called by Python and the return value is weird, but everything else in
the middle is like straight C. So you don't really have to think about what's going on. The GIL will
protect you from like race conditions, all that kind of stuff.
Yeah. And actually one of the things I love about this article is that it's using a fairly simple
example so that you're not distracted by the example. It's just the boilerplate junk that you got to
learn about. Yeah, absolutely. Which is, you know, the, probably the thing you don't know,
even if you know C, right? Yeah. And it says also, there's a few other things that are necessary if
you actually want to use this code and not just write it and compile it and see, because you have
to write a definition for your module in C and the methods that it contains. So there's a few C
functions that you call there, and then you have to build it for Python, which you basically create a
setup.py file and use just utils and it will compile and create the right base library and
install it for you. Okay. Pretty cool, huh? Pretty cool. Yeah. One of the issues with this is,
is that people that have to, a lot of times when you need to do this, it isn't a hardcore C compiler
person or a hardcore Python, CPython person that needs to do this. It's just your casual user that
happens to have a use case that they need to connect Python to C. And so this is great.
Yeah. And it's super approachable. And like you said, the examples are pretty straightforward.
Obviously you're writing C, which puts you in a different category of hard, right? I mean,
free malloc, pointers, pointers by reference, like all that kind of stuff that you learn when you learn
C, but that's the world you got to live in when you go down and you don't, you know, take the blue
pill or whatever it is.
Is the blue one the good one? I think.
No, I always forget. I know that there's a pill that's good and there's a pill that's like bad.
It keeps the facade, but yeah, probably the, I don't know.
Do you know what else is good?
Documentation.
No, Python 3.8. Python 3.8 is good.
But also Python 3.8.
For even the URL, sorry.
Python 3.8 dropped just this week. So it is no longer beta. It is final. And you can download it
from the front page. The default is Python 3.8.0 now when you download it. So yay.
Yes, that's awesome.
We've talked about a lot of stuff. We've, on this podcast, we've talked about things going into 3.8.
Like the walrus operator, of course, that's come up a lot of times. Those are assignment expressions,
positional only parameters, and fstrings. Fstrings have the little equal signs so you can debug with
them easier.
Right. Fstrings have been here since 3.6, but now they have this like self-documenting
short print statement thing, right?
Yeah, and it takes longer to describe than to show, and it's cool. What I wanted to highlight
is the what's new in Python 3.8 document that came out from, that's at docs.python.org. And it's a
really great summary of all the stuff that's in 3.8. It does have all of those new things,
that all those big hitters, but it also has some stuff that I was surprised by that I hadn't heard
of before. One of them is, we've talked about a lot of async stuff, and now you can type
python-m async.io, and it launches a async native REPL.
That is so cool, and I had no idea that that was there. I guess it would have been a pain in the
butt before to like work with async stuff over there in the REPL, right? So...
Yeah, I guess. Now you can just, because I often drop into the REPL to try something out,
now you can try out async stuff in there. So that's cool.
Yeah, that's super cool.
A couple other things that'll just help you while writing Python. A couple new warnings and
messages for things that you might do wrong. So when you're not supposed to compare, use is or is
not to compare non-objects, like strings or integers or something. It's just like, if x is 3, don't do
that. But apparently the warning wasn't very good, and so now the warning is better. It tells you to use
double equal or not equal. So that's cool. And then one of the things that I often get, because I do a
lot of parameterized testing, is if you've got a list with tuples inside, or, you know, basically a list of
lists or a tuple of tuples, and you forget the commas between some of the things, because maybe they're on a
new line or something. The warning was weird before, but now it is a more helpful message.
So I love things like that.
Yeah, you know, it drives me crazy if those are strings. Like if you're creating like a JSON document
or something like that, or a multi-line, like a list of strings, you forget a comma, it just concatenates
them, even though they're on separate lines. I'm like, oh, really? That's the default behavior?
But I understand where it comes from, but it drives me crazy.
That probably still there.
Yeah, yeah. I don't see how you would fix that without changing what that means.
Yeah. This one, it took me a while to get my head around, but I didn't know that this was an issue
before. Iterable unpacking, so if you like packed a bunch of stuff into a variable, you can unpack it
with star variable name. You can't return that in a return statement, or you couldn't before out of a
tuple. So you had to put parentheses around it before you return it. But now that's gone away,
you can just return it.
Yeah, there's a lot of good stuff in here, actually. And you just did an episode on it,
didn't you?
Yep. Episode 91 of Test and Code. I just read through the entire article, and it's still just 20 minutes.
I didn't read through everything, but it highlighted all the stuff that I thought was cool.
Super.
You know, something else that's cool is DigitalOcean.
I love DigitalOcean.
This episode is sponsored by DigitalOcean, and Python Bytes infrastructure runs on DigitalOcean,
thanks to Michael putting that all together. And it's quite solid, and we're super happy with it.
But did you know that not all web applications and services have the same memory and CPU demands?
It's shocking, isn't it?
Shocking. Anyway, so DigitalOcean has embraced this diversity in their droplet structure, which is cool,
with the ratio of memory to CPU powers in droplets. The general purpose droplets have a ratio of four
gigabytes of memory per CPU, and you can scale those up. They added, not too long, a couple years ago,
I think, CPU optimized ones. So they doubled the number of CPUs per the amount of memory,
and that's great for CPU bound tasks. But there's some applications like high performance databases or
in-memory caches or data processing of large sets that a lot of memory might be a really great thing.
So there's now a memory optimized droplet that reverses that structure and makes it like eight
gigabytes of memory per CPU. It's pretty cool.
Yeah, very cool.
Yeah. Use the right kind of droplet for the right service that you're using,
and try it out at pythonbytes.fm/digitalocean, and they'll give you a $50 credit for new users.
You and I have mentioned that folks should put legacy Python where it belongs in the past.
Last time we spoke about 35 million lines of Python code at JPMorgan Chase and their
journey to work on that. And that's all interesting. But we just recently got this announcement from
the UK's NCSC, the National Cyber Security Center.
Wow.
Yeah. And they're warning developers of the risk of sticking with Python 2, particularly library writers.
Okay.
That's interesting, right? That they actually go so far as to call that out as a thing.
So they say, look, this could be like, basically, the companies that continue to use Python 2 past its end
of life could be like tempting or setting the environment for another WannaCry or even an Equifax incident.
So Equifax was a horrible data breach. Basically, it's one of these companies that gathers up so much private data.
Like, they know stuff about my financial past that I have forgotten and don't even know, right?
They go, did you know you had this account in California? Like, I did? Oh, okay. Well, I guess I do.
Right. They know all of that. And it was broken into. Why? Because there was a vulnerability in
Apache Struts, which is an open source framework. People at Struts are like, guys, this is super bad.
You just have to send like a special ACRP request to the server and it's owned, right? Well, the folks at
Equifax got that message, but they didn't really want to get around to like upgrading it to the new
version because, hey, it's kind of hard to upgrade this thing. It's like a new version, which probably was
old and was slightly incompatible or something. Anyway, that's where Equifax came from is running
an old version of one of these frameworks, not Java itself, but like the web framework on top of it.
Anyway, there's some cool quotes in here. They say, if you're still using Python 2x, it's time to put
your code to Python 3. If you continue to use unsupported modules, you are risking the security of
your organization and data as vulnerabilities will sooner or later appear, which nobody's fixing.
Okay, that's one. One interesting quote. Another one is, if you maintain a library that other
developers depend upon, you may be preventing them from updating to 3. And by holding back other
developers, you're indirectly and likely unintentionally increasing the security risk of
basically all the computers in the world.
Yeah.
Yeah. So, I mean, we've said this before, right? You and I have said this, but if the NSA or the
NCSC, they come out and publicly call out Python 2 like this, well, that maybe carries more weight
than Python bytes. Not that we don't carry some weight, I'm sure.
Yeah. It actually makes me think though. Let's say you have a library that now works on both Python
2 and 3 and somebody else is depending on it. And they're also depending on another library that is
2 only. They're going to stick with 2. Yeah.
But if, like, for instance, you could push them if you, like, stopped supporting Python 3 or Python 2.
It's a good question. Like, in six months, do we have a obligation to actually cut Python 2 out
of our libraries? I mean, I don't have any libraries people care about, but...
Maybe? To force people to upgrade? Maybe you could do some help?
Yeah. Most of these changes have been more self-serving or self-centered, right? Like,
NumPy and Django, all those folks dropped Python 2, not because they're like, we're going to fix the
world, but, like, we don't want to maintain this stuff. We want to just move forward and use the
cool features, and we can't right now.
Yeah.
Yeah. Pretty cool.
I guess one other kind of interesting thing to call out from this report, article, whatever you call it,
is that they said that Python's popularity makes updating the code imperative, which I thought was
pretty interesting. It's like, Python is so successful. It's so broadly deployed. Like,
we can't just ignore this. It's not like Adobe Flash. It's now running an old version. We should
deal with it, right? It's like, this is one of the really important parts of the computer infrastructure
that they called out. So, yeah, I mean, there's got to be other places where we get this kind of news,
right?
So, I got a notification from a guy called Sebastian Steins, and he put up a, it's basically
a Hacker News lookalike site called news.python.sc. I don't know what sc stands for. Yeah, it looks a
lot like Hacker News, but it's just got Python stuff on it, and it's pretty neat. So, I thought, oh,
that's cool. We should talk about it. But one of the neat things about it is he put it all together
relatively quickly in like a week or so, and it's built on Django, and all of it's open source. So,
you can take it and look at how it's done and everything. Plus, it's up, and it's live, and you
can post stuff. It's neat. And I thought, yeah, maybe we'll cover this. And then while I was thinking
about covering it, we got like two or three other people tell us about this new news site. So,
I think people are using it. It's kind of fun. What do you think?
I like it. It definitely looks like Hacker News, but more Pythonic colors. And, you know,
looking through this, these are all legitimately interesting things here. I'm like, yeah, oh,
yeah. Well, I read about that. That was cool. And, oh, I didn't know about that, but interesting. Yeah,
I feel like this is great.
And even if it doesn't take off, I think it's cool to have an example of a working model
of simple with like people being able to vote things up and down. And that's kind of a neat,
kind of a neat model to say, there's a working website, a working user model. How can I emulate
that in Python?
Yeah, it's super cool. I'm definitely going to start checking it out for, as one of my news sources,
in addition to Redis and Twitter and other places.
Yeah, like we don't have enough to do.
I know. Now you just gave me work, man. Come on, it's homework.
So you've heard that most people are moving to the cloud and data science is moving to the cloud.
There's all sorts of interesting stuff happening up there. But, you know, a lot of times this type
of work, especially training like machine learning models and stuff is super, super intensive.
So if you've got like a laptop, some of the GPU processing and other really interesting things
are inaccessible to you. Like, for example, my MacBook is super killer, but it's got, you know,
like 12 cores if you count the hyper threads and it's got 32 gigs of RAM, but it has a ATI, not a
NVIDIA graphics card. So you can't use CUDA on it, for example, right?
So what do I do? I go to the cloud. Well, if you're really into deep learning and you really want to do
like data science with GPUs, there's this place called Lambda, this company called Lambda,
that is creating these deep learning workstation servers and laptops. Have you heard about this?
Huh? No. Just to be clear, this is like a super commercial product, right? These are like
servers that you buy and we have no, this is not like a product placement. I just ran across this
and thought, dang, this is interesting. So I thought I would just talk about it. So they're selling GPU
accelerated TensorFlow, PyTorch, Keras, and other types of pre-configured machines. Just, they say,
just plug in and start training. You're good to go. And they talk about how you can save a bunch of
money, right? You don't run on the cloud. The cloud can save you money for short work,
but if you got to do it over a long time, it can get expensive. So they offer a TensorBook,
which is a GPU training available laptop, capable laptop for $2,900. That's a pricey laptop, right?
Yeah.
Actually, it's less expensive than my MacBook, but still. But if you were going to do GPU stuff,
you know, this is a really cool option to be able to do it on the go or be mobile. Then they also have
a workstation, which is called Lambda Quad, which has four GPUs in it. And this one is $21,000.
Okay.
Okay. That's a lot. But if you go and grab a second tier GPU enabled EC2 instance,
specifically a P3-8X large, that's over $12 an hour, which comes out to close to $9,000 a month
in cloud bills. If you were to run it all the time, I mean, obviously probably not all the time,
but so, you know, like $21,000 is a lot, but a $9,000 monthly bill for AWS is also a lot.
Yeah. It's something to pay attention to is just as your bill starts getting bigger,
maybe a dedicated hardware makes sense.
Anytime I run across something like this, if it were Alien, or if it were gaming laptops,
or like the Apple MacBook Pro or whatever, it's like, all right, well, what if you're all in?
What if you turn all the knobs to 11? What, like, what could you get? So they have this thing called
the Lambda Hyperplane, which has eight Tesla V100 GPUs. And it starts at, it's not the final price.
It starts at $114,000.
Nice. That's without the pinstriping.
Yeah, exactly. That's not even the leather bound keyboard or whatever. I don't know.
Anyway, if you're into deep learning and you need GPUs for computational stuff, data science and whatnot,
this is actually pretty cool.
Yeah. Also, I'm sure there's applications where you really don't want to use the cloud. You want
to use in-house computers and not go out, or the connection is bad. You're sticking some data in
the middle of nowhere or something, and you can't get to the internet.
Right, right. If you've got terabytes of data, like, that takes days to upload. So,
maybe it's better to just run it locally. Who knows?
Yep.
Black has been a big hit.
Yeah, I like Black.
Yeah, for sure.
A lot of people do.
Oh, yeah.
One of the things, so I ran across an article. It's not a new article, but it's all still relevant.
It's auto-formatters in Python. And big shock, Black's in there. But one of the things I
liked about it is they spent a little bit of time talking about why you want to use Black
or something else. And I'm finding this more and more as a team lead that just, it's not great to
have, like, if you're doing code reviews, you don't want to have, like, style be part of the code review.
Yeah.
It's way better to have a tool just dictate the style. And so, people can argue with the tool
instead of arguing with each other.
Yeah. It's like, if the code review, the people there, I'm sure they feel like, well, I have to
make a constructive or critical comment about something. It shouldn't be, why are you indenting
like that? Or why is there not a space by those commas? Like, that's the stuff machines can agree
upon and just do for us, right? Like, have architectural or algorithmic conversations, right?
Yeah. You should be using three double quotes there instead of one. So, get off the style
police and use an auto formatter instead. I love Black. A lot of people do. But there's reasons
for some people, like an established code base or other predefined style guide, that maybe it's too
much. It does do things that sometimes I don't like it to do. So, there's a couple other options.
And this article talks about auto PEP 8 and YAPF. Now, auto PEP 8 is essentially just, it just does
PEP 8. It's, or uses PyCodeStyle, the utility, to detect PEP 8 violations and just change the code.
You can do both with it. It does less than Black, but it doesn't do much more. So, if that, if really
you're just trying to stick to PEP 8, maybe that's, yeah, that'd be better to use. And the other end of
it is YAPF, which is a tool out of, and I don't know how to say that, YAPF? It's probably yet another
Python formatter. Yeah, it probably is. It's a Google tool. I think it's Google. I think it's good if you
want, it's got a lot of knobs and dials, a lot of customization. So, if Black doesn't have enough
controls for you and you really want to tweak it to be your personal company's code style, this might be
great for you. In the documentation, it says it takes away some of the drudgery in maintaining your code
code. And what, just ultimate goal is to code, is that it produces as good a code as that a programmer
would write if they were following the style guide. That sounds pretty good, honestly. One of the
interesting things, I was researching this story, is I didn't know this about Black. After it's changed
your code, it does a check to see if the reformatted code still produces a valid abstract syntax tree
that is equivalent to the original. That's pretty cool. I didn't know it did that.
Yeah, that is cool. So, run it through the Python
parser and turn it into
bytecode and then just see if the essence is the same, which
yeah, I mean, because you don't actually want to change
the meaning of the way the code actually gets interpreted. It's just formatting, right?
So, the meaning change is like, well, that might be a problem.
Yeah. The other thing I wanted to highlight this
article for is it took a few code examples and just did the, what does Black change it into? And what does
Yapf change it into? And what does Autopepate change it into?
Oh, that's sweet. I like that.
Yeah.
Very, very cool. All right. Well, that's all of our main items. You got anything else you want to throw out there while we're here?
No. You?
Yes. A couple things. I'm getting excited for PyCon US. It's earlier this year, in April at some point, I'm guessing. But the announcement I want to make is that the applications for financial aid are open and they'll be accepting them through January 31st, 2020. So, 30 days into a world with only Python 3. The Python Software Foundation and PyLadies are making this financial aid possible and check it out.
Yeah. So, like PSF is contributing $130,000 towards that. And yeah, it's pretty good. So, if you're thinking, hey, I would really love to go to PyCon and make some connections, kind of new to this world, use some networking and learn more about it, but I just can't justify the expense or afford it, maybe do that.
Yeah. Nice. Indeed. Indeed. And I'm working on some new courses. I got one that's all done and recorded, just getting edited. Another one, I spent like six contiguous hours recording videos yesterday. That doesn't sound like a lot of time if you haven't done it, but six straight.
Eight hours recording, that's a lot. So, I'm really, really excited about what's coming out. We'll share more with it when I can.
Very exciting.
Oh, yeah. Now, sometimes we have really short jokes. I see that you have one.
We got a short joke that was contributed by Eric Nelson. Thanks, Eric. It is a math joke. The joke is, I is as complex as it gets. JK.
The letter I. Yeah.
I love it. I love it. I studied a bunch of complex analysis and things like that when I was doing math and yeah, I like it.
Yeah. We have another one too that it's long.
It's long and I'm not going to be able to do justice to it, so you have to check this out. So, you know the song American Pie, right?
Yes.
I drove my Chevy to the levee, but the levee was dry. That sort of song?
Yeah. You can sing it.
No, no, I can't sing it. I could recite it.
If I sing it, it's not going to be singing. It's going to be something else. There's another one. One of our listeners, I only know his username on Reddit, I'm afraid. I can't find the tweet in time.
Wait. It said, hey, you inspired me to write this song called American Pie, American P-Y, and it's basically the story of like Legacy Python done to American Pie, the song.
Yeah, it's pretty awesome.
It's really, really well done. I'll just like recite a little bit here one of the verses.
So, bye-bye to your Legacy Pies. Made decisions about divisions, so you'll have to revise. And you decode official, it's not a bunch of bite lies. Singing, that'll be the day it dies. That'll be the day it dies. It's really good. Yeah, people should check it out. If somebody can perform this and give it to us, he's given us permission to take that and put it on the air. If it's good enough, man, we'd love it. That'd be awesome. I cannot do this.
I want somebody to sing it because it includes the phrase, I was a crusty old fart coding guy.
Yes, I know. You could be a YouTube sensation if you just take this chance here. Jump on it.
Yes, and if you do, let us know.
Yeah, for sure. Let us know. That'd be awesome. All right. Well, yeah, this really nice song and a nice job there. Well done on that. And Brian, thanks for everything. Thanks for being here.
Thank you. Yep, you bet. Bye. Bye.
Bye. Thank you for listening to Python Bytes. Follow the show on Twitter at Python Bytes. That's Python Bytes as in B-Y-T-E-S.
And get the full show notes at pythonbytes.fm. If you have a news item you want featured, just visit pythonbytes.fm and send it our way.
We're always on the lookout for sharing something cool. This is Brian Okken, and on behalf of myself and Michael Kennedy,
thank you for listening and sharing this podcast with your friends and colleagues.