When you think of popular Python packages, what comes to mind?
There's a good chance that this week's guest, Kenneth Wright, wrote the package you just thought of.
He's the author of some of Python's most popular libraries, including Requests, Records, Maya, and Pipenv.
I got a chance to catch up with Kenneth at PyCon 2017 this year. Here's the story.
This is Talk Python To Me, episode 115, recorded live at PyCon on May 20, 2017.
Welcome to Talk Python To Me, a weekly podcast on Python, the language, the libraries,
the ecosystem, and the personalities.
This is your host, Michael Kennedy. Follow me on Twitter where I'm @mkennedy.
Keep up with the show and listen to past episodes at talkpython.fm, and follow the show on Twitter
via at Talk Python.
This episode is brought to you by Codeship and Hired.
Be sure to check out what they're offering during their segments. It really helps support the show.
Kenneth, welcome back to Talk Python.
Hi, Michael. Thanks for having me.
It's really nice to have you back. You always seem to have so many amazing projects.
Oh, thank you very much.
And your pulse on what's going on in the community. And we happen to be like right in the center of it.
Yes, we're here at PyCon. This is my favorite week of the year by far. It's the highlight of my life.
It's definitely a highlight for me as well. And it's the fact that I can drive to it and not fly to it.
Oh, where do you live?
I live 15 minutes from here by car.
Oh, I'm very jealous.
So that makes it quite easy.
It won't be that way next year.
No, next year is going to be a four-hour flight with no... If I'm lucky, right?
Yeah.
It's all good, though. I'm definitely making it there.
We'll have the Rock and Roll Hall of Fame. It'll be worth it.
It'll totally be worth it.
And, you know, there's a bunch of people that wanted to go to this one, and it's sold out, right?
Yeah, it sells out every year.
So if you guys are listening and you're like, this sounds so fun, I want to come, but it's sold out, get your tickets early.
Yep, always get your tickets early.
PyCon sells out every year.
Every year. Okay. You don't need too much of an introduction, but you've done a ton of amazing projects that we're going to talk about.
But tell people, what do you do day to day? Like, who are you?
My name is Kenneth Reitz. I'm most well known for being the creator of the Request Library, and I work for Heroku.
I'm the Python overlord at Heroku, so I'm the language owner of Heroku.
It's not the language owner of Heroku. The language owner of Python at Heroku,
meaning that everything that's related to Python, basically, that is about the Heroku product is something that I am in charge of.
I see. So if I want to deploy my Pyramid or Flask web app...
Yes. It's my job to ensure that you can.
Or decide if that's appropriate.
Yeah, exactly.
Okay. Yeah, that sounds like a pretty fun job. I've never really had a position that was at an overlord title.
Oh, I gave it to myself.
Yeah. Well, look, that's all right.
They asked me what they wanted on my business cards when I started, and I said Python overlord, so it stuck.
With the fact that they let you do it, that's awesome. That's really cool.
My official title is Senior Member of Technical Staff.
I see. That's way less fun than overlord.
I know, I know.
But no, it sounds like a pretty fun project. You like working at Heroku?
I do. I've been there for five and a half years, and I'm not planning on leaving anytime soon. It's a great company.
Yeah. You don't think this cloud computer thing, it's a fad?
No, I'd say I think we're good.
I think it's going to be around for a while, right?
Yeah, definitely. Platforms as a service is something that I think only more and more people are going to be using.
And infrastructure as a service, I think, is probably going to change a lot over the next several years, because things like Docker are coming out and becoming more ubiquitous, and there's a lot of technologies and innovation happening in that space.
And I think that platforms as a service are going to provide a stable platform for people to kind of keep their heads on straight in that world.
Yeah, that sounds really interesting. I like Docker, but I feel like we're kind of, like, Docker is an intermediate evolutionary step of something else.
Yeah, I know exactly what you mean.
Yeah?
I haven't used Docker too much. I've only done, like, little toy experiments with it or building example repos.
To me, it feels like a response to a need that exists, but it's, like, not the answer. It's just, like, the first answer.
Right, like, we have these VMs. They take too long to set up.
Too long to start.
The cache is ridiculous on your machine.
If you use Docker in any capacity, you should check your disk cache for Docker.
There's a command to do it, and it'll be, like, 30 gigs.
You know, there's a lot of room for improvement, but I have to give them a lot of credit.
The Docker format client is stellar.
It's a very nice client, and they did a lot of work to build it the right way.
Nice.
So do you guys have Docker support at Heroku?
It's not in GA yet, meaning it's not in general availability, but we do have it in beta.
Okay.
Yeah, and it's available.
You can use it in production if you'd like, and I recommend people doing that, especially if you're using scientific applications.
It works very well for that on Heroku.
So you can app get install anything you want.
You use Conda, for example.
Right, so you can have a lot of the machine learning stuff that's got lots of weird compilation all set up right on your Docker image, right?
Yeah, exactly.
Okay.
Yeah, very cool.
So maybe tell us, what's it like to deploy a Python web app to Heroku?
Git push Heroku master.
Sounds all right.
Now, so you have a requirements.txt file that you use to specify your pip requirements, and that's how we know that it's a Python app.
Okay.
And we install those requirements, and that's effectively it.
Then you have a proc file, which is standard on Heroku, which you use to say what your entry points are for your application.
So you do, like, web colon space command to start your application.
Gotcha.
Okay. Yeah, sounds very cool.
And that's all you need.
Yeah, so that's maybe a good segue over to the projects that you're working on.
Like, you've been doing some stuff trying to make virtual environments or environments in general better, right?
Yes, yes.
I'm working on a project right now called pip Env.
It's had, like, 40 releases in the last three months.
It's been quite a bit of work behind it.
It's relatively stable.
All right, so what problem is it trying to solve?
It's trying to solve, try to be as succinct as possible.
It's trying to combine the workflows of pip and virtual M together, as well as make it as easy as possible to enable deterministic builds and solve the UX problem of requirements.txt.
Because requirements.txt can be used deterministically, where you just specify all your requirements and all your pin versions.
Yeah.
And it can also be used in a more human-friendly format, where you just put the things that you want, and you may or may not put version numbers in there.
Right.
And sometimes it's fine, and sometimes it's not.
So there's a new project from the PyPA, the Python Package Authority, called pip File.
And it's a new standard that will replace requirements.txt.
And it's kind of like a YAML file, is that right?
It's a TOML file.
TOML file.
Yes.
YAML was discussed, but TOML is what we ended up settling on.
There are two components to it.
There's a pip file, which is the human part that you put in the things that you want, like Flask.
And then there's a lock file, which is JSON.
And it is generated by a machine and can be parsed by machines.
And it has the deterministic list of all of your packages that you want to have installed, including all the versions.
So if you had Flask in your first file, the second file would have Flask, WorkZug, isDangerous, Click, etc.
I see. That's cool.
So does it capture the version?
Yes, absolutely.
So it captures the version and the transitive closure of your dependencies.
So you say the top-level dependencies, and it gets the lock JSON as all the dependencies.
Correct.
And you have two groups.
You have default and develop.
So you can say that there are things that you just want for development and things that you just want for production.
Right, like pytest.
Exactly.
Probably not a good reason to deploy that.
In production.
Yeah, but there's no reason.
It's not harmful.
So if you're doing something like Selenium or Django toolbar or something like that.
You need the debugger toolbars that give you extra access.
Yeah, exactly.
Maybe don't put that in the plain one, right?
Exactly, exactly.
So that's the new work that PyPA is doing.
So yeah, that's what pipfile is.
And so what pipenv does is it allows you to use...
So pipfile is going to be put into pip in the future.
Do you have any idea of the timing?
There's a man named Donald Stuffed who is solely responsible for almost all things Python packaging.
He is Mr. Pip.
And so he is the gil that is responsible for the contention lock on that project.
He's doing good work there.
I've contributed to helping it come along, but he wants to rewrite it and make it more functional and less object-oriented.
And then eventually we are going to land it in pip proper, so pip will support it.
In the meantime, you can use it in pipenv.
So you can use this great new functionality today.
And what pipenv does is it takes away the brutal frustration of having to create and manage virtual environments.
In addition, it also allows you to use the pipfile and the pipfile.lock, and it gives you a lot of other niceties.
For example, I was inspired by using Composer for a few moments when I was helping someone with some PHP stuff,
which is not something I do, but it was something I was helping a new developer friend of mine walk through a tutorial.
And we did Composer install something, and it said adding that package to Composer.json.
And that was when the light bulb struck.
I thought that was a great idea.
Yeah.
So when you do pipemp install requests, it will automatically add requests to your pipfile for you.
So you don't have to go and manually curate this file.
It'll add and remove packages from it for you.
So the file exists basically as you pip install things.
Pipemp install things.
Inf install things.
It will find that file and update it automatically.
Precisely.
All right.
And you can lock it, and then you can also specify if you want a Python 2 or Python 3 virtualenv.
And it's really nice.
It put a lot of work into the UI, and it has some nice colors and Braille spinners, and it's a very polished product.
Really awesome.
And so that's out now, right?
People can find it.
Yes.
What's the path?
How do they go get it?
Well, you can pip install pipemp.
It's very meta.
And also, it's available at pipemp.org, and there's a nice little animated GIF that'll, or
GIF, I'm not sure what the official terminology there is.
You're going to start a GIF GIF fight.
I don't remember.
I'm too tired.
It's PyCon.
Yeah.
I actually don't know which one's proper.
Anyway, there's an animated image file that allows you to see it in action.
So yeah, pipemp.org.
You can check it out.
Nice.
And are you looking for people to contribute to it, or is it kind of doing its own thing,
or what's the status?
It's relatively stable.
What I'm looking for right now is as much feedback as possible, so I can know how to
iterate next.
Awesome.
So people should just try it out.
I want everyone to start using it, because I think it's in a great place where it could
become the next virtual M wrapper, effectively, the thing that everyone uses to manage virtual
M's and be their go-to tool.
And I think it's actually ready for that.
So I just kind of want to market it and get people to know it.
And then once it gets enough, the snowball starts rolling, then people use packaging tools
because they're told to by other people.
So it needs to be...
The tutorial says type this.
Exactly.
So that's...
It'll be a snowball effect, and it'll...
That's the goal for the project, is to become the next de facto standard, and to have all
the best practices baked into it so that you don't have to worry about them.
Yeah.
It sounds really nice.
It also sounds beginner-friendly.
Yeah.
It's very beginner-friendly.
It's also very advanced user-friendly, too, because it just does all the stuff you would
do by hand for you.
Yeah.
It's just the beginners don't know what the magic is, and the people who have been doing
it for a while are like, oh, it's doing these three things, or whatever, right?
Yeah, precisely.
Precisely.
But if you're a beginner, or you haven't used Virtual Lab and stuff like that, it just works,
and it tells you what to type next.
You just type pivm shell, and it gives you a shell that has your new Python interpreter
in it.
And it's much more approachable, and it works on Windows, and it's just a lot of
UX polish into the whole process.
Yeah.
It's good that it works on Windows.
That means it's really easy to recommend for your tutorial and your whatever project you're
trying to get set up.
Okay.
So that's your new virtual environment dependency project.
What else are you up to since we talked to you about two years ago?
I've written a couple of libraries that have been interesting.
The one that I want to talk about the most is called Maya, which I think is very interesting.
How do you spell it?
M-A-Y-A.
Maya.
Yeah, yeah.
Like the Hindu philosophy of the illusion, because time zones are an illusion.
So it is a daytime library that kind of turns time zones on their head, because when you're
dealing with time zones, you have to deal with localization if you're doing any time zone
algebra normally.
So that's if you want to do any math to calculate this time zone versus that time zone, which is
something that you have to do a lot if you're dealing with timestamps.
Effectively, you're already you're parsing a website.
Yeah, for example.
What are we going to say?
I was going to say, like, if you're already dealing with time zones, you're like, you're
a little one step down that path as well of like, well, yeah, you can always hack it to
work.
But if you're trying to do things the right way, I actually cannot figure out how to do
it the right way without using it without building this library.
So the problem for me was writing a little thing with like an RSS feed and it scraped the
time from a website and I told it what time zone it was in and like it would report
back that the RSS feed was being published at like two in the morning and it was like that
was not the right time.
I wanted to be at midnight.
So I wanted to fix it the right way at the standard library and I and I worked on it for
like four hours on like three, like four lines of code.
And, you know, and I'm like someone who knows Python pretty well, you know, is it's like
not, it shouldn't be this difficult.
You should be able to compare two times.
Come on.
And I spent all that time and then I finally got it to work and then I deployed it and then
it didn't work into production because the server had a different locale than my local machine
because the time zone algebra is different when you have different locales.
And that was what the straw that broke the camel's back.
So I see.
It's one thing to have just time zones.
It's another to have German time zones versus U.S. time zones or something like that, right?
Precisely.
Yeah.
When I'm doing math on time zones, I don't care what the machine, where my machine is located.
It's the same everywhere.
It doesn't matter.
That should not be taken into account when I'm doing time zone algebra.
So Maya exists to be, have the philosophy that your data should always be in UTC all the
time.
You can import it from another time zone.
You can export it to another time zone.
You can do that very easily.
You can generate date times with it, either naive or non-naive.
What's a naive date time?
A naive date time is a date time object that does not contain a time zone object.
Yeah, sure.
So in Python, you can't be kind of like...
Which is surprisingly difficult to do.
That's not like a one-liner.
That's like an eight-liner in Python.
Which is crazy.
If you've got one date time that has a time zone.
But with Maya, it is a one-liner.
Yes, of course.
Yes.
And you've got a date time with time zone and one that doesn't.
You can't even subtract them, right?
They can't be...
I'm not sure what would happen.
I think it would say it's incompatible.
Yeah, I think they're kind of incompatible with each other.
Yeah.
So the way that you should approach this, if you're ever building an application, is that
all of your data should always be UTC all the time.
And you should know that.
So it should be time zone aware.
You should say it's always UTC.
You should never store it as anything else other than UTC.
And UTC has its problems because UTC is off by a second now and then.
But that's okay.
We're building...
This is for humans here.
This is a date time for humans.
Right.
So the idea is...
We're not working on particle colliders.
Yeah.
This is for parsing websites and dealing with dates on websites and things like that.
This is not for scientific uses and stuff like that.
So it's all based on epoch time.
And it does the math that way, based on a number of seconds since, I don't know, January 1st,
1970 or something like that.
Yeah.
And it doesn't use any localization for any of its time zone algebra.
And it works very well.
And it allows you to just pass in...
It has two methods for importing stuff.
It has dot when.
And you can pass in there any English, any human form of a date.
Okay.
And it'll give you back the Maya date time object.
That's awesome.
Like, give us some examples.
Like, a week ago, for example.
Because that's something that you see on a website.
Yeah.
Or you could just, like, the way any human would ever write a date in any way.
It'll...
Two hours from now.
Yeah.
Two hours from now works.
You could do, like, 2, 4, 3, you know, or 2, 4, 14, something like that.
That works January 15th, 2015, stuff like that.
And you can parse all these?
And then there's dot parse.
And dot parse gives you...
It will read all known standards for machine parsable dates.
And it'll give it to you in the Maya date time object, which is in UTC.
And then you can export it from there to a date time object.
And then it has methods to import and export from all the major RFCs for all the date time objects.
So I'm very proud of this library.
I think it's great.
And I think everyone should use it.
It's awesome.
Yeah, yeah.
I've definitely checked it out and done some stuff with it.
You're right.
Those are just surprisingly hard problems.
And I don't know why.
It took me, like, a surprisingly lot of thought and work to build this library.
And it shouldn't have.
Like, it took me a good three days of solid work to build the prototype of it.
And it...
So just...
I had the API in my head of what I wanted.
And then, like, to actually get it working properly...
All variations.
...by using the proper APIs and stuff, I had to look up so many Stack Overflow questions and stuff.
And that's the pain I'm trying to help people avoid is to ever have to ever do any of that work.
You're like, okay, we should solve this once and for all and forever?
Precisely.
Awesome.
All right.
Well, thanks for that.
Absolutely.
So I want people to give that a try, Maya.
If you ever have to deal with date times, especially if you're parsing websites or dealing with data from another source that has a date time, you should check out Maya.
Cool.
Yeah, so that sounds really cool.
There's two other projects that I know of that kind of in this area, I know they're not necessarily solving the same problem.
One is called Arrow.
Yes.
Are you familiar with Arrow?
And then the other is Python DateUtil.
Like, can you just maybe say...
So Python DateUtil is a dependency of Maya.
Maya also uses something called Pendulum, which is similar to Arrow.
All right.
So it's kind of like this competition in this space, but we're all friends and we're building each other up.
So I'm using Pendulum to do some of the...
I was doing my own date parsing with another humanize or something.
I have like five or six dependencies that do a lot of this stuff.
And I just wanted an API that like put all the tools together and works great.
So Arrow, I'm not overly familiar.
I had to look at the readme to know which one's which off the top of my head.
Yeah, I always confuse them myself.
They all weren't what I wanted because I really want this UTC is king slash queen like mentality in my head of that.
Like it's always your data should always be UTC and that you can import and export to other things.
And a couple of them did have that, but it just wasn't as solidified as I wanted it to be.
So, okay.
So, yeah.
Yeah, that's a really cool project.
I've already played with it.
And I was talking to someone here, David Golden, and he's talking about adding a relatively complicated class to add times ranges.
So we may get the ability to do some relatively complicated time range algebra.
Right.
For calendars, like are there conflicts?
Exactly.
That's where it's coming from.
Yeah.
Time conflicts for calendars for like 25 people and stuff like that.
That's what he wants to solve.
So we'll see if we get there on Monday.
That'd be an awesome add-on.
So you're going to do that for the sprint?
You're going to work on that?
Potentially.
Yeah.
If it's a good fit.
Okay.
Yeah.
Very nice.
This portion of Talk Python To Me is brought to you by CodeChip.
Try CodeChip Basic, a free, simple, out-of-the-box, continuous integration service in the cloud.
Thousands of customers use CodeChip Basic every day.
Its pre-installed CI dependencies make testing your software and deploying it work out-of-the-box.
The average setup time for CodeChip Basic is less than three minutes.
CodeChip Basic comes with a free plan that grants 100 builds per month, unlimited projects, and
unlimited users.
Do you run an open source project?
Those are always free on CodeChip.
And they just improved their Python support.
So give CodeChip a try and visit talkpython.fm/CodeChip to learn more and sign up for
free.
That's talkpython.fm/CodeChip.
So another project that you're working on that I think is really nice and has a very,
take something kind of overly complicated and make it simple is Records.
Yes.
I don't think we talked about Records before, did we?
I don't remember.
I don't believe so.
I think I wrote it after we talked last.
Yeah.
So yeah.
Why don't you tell everyone about Records?
Actually, someone came into the booth, my booth, and said, hey, tell me about Records.
You know, I heard you talking about it.
It's really cool.
Show me a demo of Records.
Oh, cool.
Isn't that cool?
Cool.
Yeah, those guys are fun.
That's awesome.
So yeah.
Tell everyone else about Records.
So Records is, so SQLAlchemy is an incredible library for dealing with SQL.
And it is really interesting when you work for an organization like I do, the way that we
deal with data is that we use tools like Heroku Data Clips.
And we use a tool called Looker, which allows you to get access in our data warehouse and
do some very advanced queries and get SQL out of it.
And I want to be able to take raw SQL that is written, that is trusted by a source like,
for example, Looker, which is generated by our business team, BizOps team, and put it
into my code.
And you can do that with PsychoPG, too, very easily.
You can just like, say, execute this SQL.
But SQLAlchemy gives you a lot of other benefits, like connection pooling.
And it has nice connection drivers.
You can figure out what kind of database engine to use on its own, right?
Precisely.
SQLAlchemy, MySQL, it just figures it out.
Yeah, and it has a very nice way of, and I also wanted to get back dictionaries of data.
So each row is a dictionary.
It's also a tuple as well as a dictionary.
It acts as both at the same time.
Yes.
Like a named tuple, right?
It's a named tuple that is also a dictionary.
It's kind of both.
It's a very cool little thing.
Yeah.
And so I guess for people, if they haven't used it, it's a little bit like just the dbapi,
too.
One line of it.
Right?
You just like...
Well, the idea is if you want to use SQLAlchemy and you want to just do raw SQL, you
have to Google a lot of stuff.
And it's like a good...
If you want to do it properly, it's like a good 25, 30, 40 lines of code.
Kind of got to break out of the object ORM side of things.
Yeah.
And then there's some niceties that you want to build into it.
And then it ends up being 100, 200 lines of code, 300 lines of code.
So I built a library for that.
And in addition to that, I also built exporting.
So I have another library that's very old called Tablib.
Okay.
And it gives you tabular data sets that you can export as Excel or as CSV or as a bunch
of formats, JSON, YAML.
So this supports that as well.
So you can take your raw SQL.
You can get Excel out of it.
You can get TSV.
You can get CSV.
You can get HTML tables.
Yeah.
Very, very cool.
So it's just basically for taking a report from SQL and sticking it into your code base.
And either using it to power your application or to generate a report.
And there's a command line tool that comes with it too.
So it's very little useful, small little thing.
Right.
It's a fun little toy.
But it's very, very powerful.
It's cool.
It seems to me like if you just want to connect, ask the database a quick question, like this
is a really nice way to do it.
Exactly.
Yeah.
And you don't want to use SQLAlchemy directly if you're doing that.
Because again, you have to look it up and it's not very intuitive.
And if you want to parameterize your queries, it's not standard.
It took me a while to figure out how to parameterize queries in a standardized way that works across
all query languages.
So I had to figure out a lot of stuff to get it to work is what I'm trying to say.
So I solved all those problems with records.
Nobody wants to be little Bobby Tables.
Yes, exactly.
Exactly.
By the time you put all the correct pieces together, it's a lot of work.
Yeah.
So exactly.
So it's easy if you're going to use PsychoPG2 or something like that to, they have good parameterization
in PsychoPG2.
I shouldn't say that.
Yeah.
If you're going to use something like SQLAlchemy or anything like that, if you're going
to be using raw SQL, it's really easy to not parameterize properly.
One little plus with the string and the variable and it seemed all so innocent.
Yeah.
So innocent.
What could possibly go wrong?
I just merged that one pull request that I got from that stranger and then suddenly all
my bank accounts were empty.
It didn't make any sense.
I don't know what happened.
This world's complicated.
So are there other projects you want to talk about or maybe just stuff that inspires you?
So interesting things in the Python world right now for me are API Star by Tom Christie.
Yeah.
So he's here not far from my booth and a little not too far from your booth.
And he's doing really cool stuff.
He's the Django Rush framework guy.
Yes, he is.
Yeah.
Okay.
So he built this thing that is a new API framework for Python 3 only.
And it uses type annotations in the first way that I've ever nodded my head in agreement.
Whenever I've seen type annotations, I was like, that's the stupidest thing I've ever seen
in my life.
Why don't want that in my Python?
But when I see him using them, I'm like, oh, I get it.
That makes sense.
Yeah.
So I think it's kind of a cool idea.
And I used it to throw together a little toy API the other day.
And the cool thing about it is that it's not Flask.
I love Flask.
Flask is incredible.
I'm a contributor and maintainer of Flask.
Flask is one of my favorite things in the world.
But it's just nice to change things up.
Yeah.
And the really cool thing about it is that it automatically generates documentation for
you.
And it's interactive documentation.
Awesome.
So.
And it uses the type annotations to do that, among many other crazy uses of type annotations,
right?
Yeah.
It's a really cool project.
And it's really early on, but it's getting a lot of hype.
So he's going to put in all this work.
And I think it's going to soar.
It might be like the next Flask for API design, maybe.
Yeah.
It's looking really, really promising.
It might be companies hiring API star developers one day.
Yeah.
You know?
Yeah.
I can see that.
It's nice to be there at the beginning, you know?
So I encourage everyone to check it out.
Yeah.
It is really cool.
I was impressed with it as well.
I'm going to have him on the show in a few weeks.
Yeah.
It's also one of those things when I'm using it, it's so different than what I'm used to
that I found it a little cumbersome.
Yeah.
So I'm not actually sure.
I'm not endorsing it necessarily.
I'm just saying that it's really interesting to me because it's something so different.
Here's something new and fresh.
Yes.
I need that.
I think I really get bored with using the same libraries all the time because they're so good,
the ones that we have.
And it's nice to get a new tool in my box that's like, oh, I could use this for something.
Yeah.
For sure.
And that's a rare thing for me.
Yeah.
It's really great.
And also, I mean, he's already been maintaining Django REST framework.
Yeah.
So he knows a lot about APIs and it's all about self-described APIs too.
So he supports VDN and HAL and all, or is it HAL?
I don't know.
He supports all the self-described API stuff.
So it's all hypermedia friendly and it's very proper REST framework stuff.
There's a project.
Well, I guess he's going to talk about it, right?
Yeah, I guess so.
You go give us the preview.
Okay.
There's a client that comes with it too.
And it's called core API.
And it's the kind of a compliment to it where any API you write with this, you can consume
with this thing called core API.
And there's a command line client for it, as well as a JavaScript client for it, as well
as a Python client for it.
And you automatically get, because they're self-described APIs that describe themselves, you get a fully,
like, you don't have to write a client for your API.
It's just already done.
Oh, wow.
And in the docs that he generates, show you how to do it in all three of those languages
built in.
It's like a really solid piece of work.
I'm very impressed.
That is super cool.
Yeah, the thing that I thought was interesting was, here's a guy who's very committed to
Django REST framework, creating a new framework.
And that caught my attention.
Like, wait, weren't you doing that?
You know what I mean?
Yeah, yeah, yeah.
And it's like, well, he can't start from scratch with Django REST framework.
It's built on a long heritage of a way of doing things.
And this is like, what if we could start again, Python 3 only right now from scratch?
And what if we take Django out of the equation?
And that's really important, because Django is not necessarily the best thing to be building
APIs on, necessarily.
It's not necessary.
Right.
It's got a lot of stuff to support rich web apps.
And you can use Django as a library.
So you can import Django into an API Star app if you really want.
And then you can just start using your models in it.
I mean, there's nothing stopping you.
Yeah, yeah.
How interesting.
All right.
What other projects?
Projects that I love.
Yeah.
I'm really in love with DocOpt.
I've always loved that.
Yeah.
DocOpt and Click.
There are two command line libraries.
I was playing with Click the other day.
Click and DocOpt are kind of like matter, antimatter.
Yeah, they're very opposites of each other.
Exactly.
Yeah.
Tell us about them.
It's amazing.
It's the opposite of aug parse.
It uses aug parse.
It turns out when you type any command in Linux and you do --help and it prints out
like the thing with the brackets and all this stuff with the commands.
Sometimes it's super complicated with lots of symbols separating things.
Yeah.
There's all these angular brackets and stuff.
And you're just kind of like, okay, I can infer what that means.
That's an ANSI standard, it turns out.
And so you just write that help string in DocOpt in Python as a string.
And then you say, parse my arguments.
And it just magically knows exactly what to do.
So you don't have to write any parsing code.
And it's just amazing.
So it's good for small things.
Like if you're writing a command line thing that doesn't have any like sub commands, maybe
like one or two sub commands, it's okay.
If it can fit into a single help printout screen, then DocOpt is great.
It's an option, yeah.
Click is fantastic and a very engineered piece of project from Armin Roeneker, the guy from
Flask and a good friend of mine.
And he, again, is very engineered.
It's like everything from Armin is in a good way, in a good way.
You can dive deep into it.
And it is, if you're going to build something that has a lot of structure and something that
has a lot of sub commands and something that has...
Something like Git, maybe?
Yeah, or PipM.
PipM uses Click, for example.
Because it has, you know, I have like five or six sub commands and they all have different
arguments and different defaults.
And DocOpt wouldn't be a good fit for that.
But Records uses DocOpt, you know, different tools for different tools.
Yeah.
Different dependencies for different dependencies.
Yes, precisely.
Precisely.
Awesome.
Other projects you've been loving lately?
This is the things that come to mind off the top of my head.
I was talking to the Beware guys that are over here.
Yeah.
I'll do, I guess, a little bit of evangelism for the project.
Cool project.
It is a cool project.
I knew about it last year, but they've come a long way.
So Beware allows you to, in pure Python with a pip installable module that is no C extensions
or C dependencies at all.
To write native Python with a very CSS inspired graphical representation of, that's how you
like manipulate the graphics is with like a CSS style Python thing.
You can render Windows forms, Coco for macOS and GTK for Linux and web, single page web apps for the web and Android apps and iOS apps, all from the same code base.
And there's no C compilation whatsoever.
And the thing that I'm excited about that I just learned about during PyCon was that they've taken that and they've made a project called Briefcase, which allows you to package up your application into an executable distributable, which is something that I used to work with quite a bit.
And there's a lot of packages out there that allow you to do that, but they're very cumbersome and they're all very different.
They sometimes work.
And they, they all, you can always get them to work.
But it's a, it's a challenge sometimes.
Yeah.
Especially if you're using something like PyQT and embedded web kits and it's, it can be very cumbersome.
So it's, I'm very impressed with the demo that I got.
So I think it's pretty cool looking.
And then I might start playing with Briefcase.
I'm really.
Because I like shipping executable distributables.
Yes, exactly.
Like that, I feel like that's one of the shortcomings we have is that we don't ship that many.
Hey, here's my app.
Double click it and look at the amazing stuff it does.
As a Python community?
Yes, yes.
I completely agree with you.
Yeah.
At the keynote here, we saw like this hockey stick growth of Python and you could see it
across other metrics and like Stack Overflow trends and other places.
And I just, I think how much steeper would that be if we had a solid desktop UI story?
Yeah.
You know, I mean, that could just like, people already love it so much.
The second you said that, I was like, I could fix that.
But I have a policy of, I don't want to build things.
I don't want to, that I'm not dog fooding.
Yeah.
Or I'm not sipping my own champagne.
I don't have any graphical user interfaces to build.
Yeah.
But I would love to, that'd be so much fun.
I would love to see that.
That camera.
I could write the documentation for that.
There you go.
That's what, it just, it's really a documentation problem.
Yeah.
Well, those guys have some pretty solid stuff.
You just have to say, this is the proper way.
And then people do listen to you.
It's really cool.
Do it this way.
Yeah.
Awesome.
All right.
So, other projects top of the, run through your, your high, high love list.
High love list.
Those are the ones that have come to mind.
Flask.
Always great.
I'm technically a core contributor and I'm going to, I've been running out of things to do in my free time lately.
Like I've just been really bored and I've been dying to figure out something to do in open source lately.
Cause I've been working on PipEmp and stuff and, and that I've kind of atrophied with like things to do there.
Cause it's kind of done.
And I have to wait until I make a new decision to make any progress there.
Cause it's, it's very stable.
Right.
That's why you need people to kick it around a bit.
And I need more users.
Yeah, exactly.
It has a nice user base, but I need the masses to come.
Well, you're used to a different level of user base.
Precisely.
Yeah.
I only have 10 million users.
I don't remember where I was going to go with that,
but what was your question again?
What, what other projects do you love that you're like, you said you were thinking about contributing to something.
Oh yeah.
So I'm might make a point to start actually being a real maintainer of flask.
I think that would be good.
Help triage issues and stuff like that.
And I would be proud to say, I did say earlier because I'm planning on doing this, that I am a committer to flask.
So I've had the commit bit for like three years, but I've, I've only merged like three pull requests.
Sure.
Yeah.
I've never really done anything with it, but I would be really proud to add that to my resume that I'm like, that I helped maintain one of the most beloved pieces of software ever.
In my opinion, I love flask.
I think it's just the best.
Yeah.
I love the simplicity of flask and frameworks, right?
One of those things though, where I, as a user, I don't have any complaints.
Yeah.
So I don't know what to work on.
I don't want to do anything to it.
Yeah, exactly.
And I'm sure if you open the issue tracker, you'd find quite the opposite, but I have as a, I don't have any complaints, so I don't have anything I want to work on.
Yeah.
I guess.
I don't have any ideas because it's one of those things that's written by someone who's a lot smarter than I am.
Armin.
Well, I think, you know, around a lot of those things, one of the places where it could open up is the whole async await stuff, but, but that's so much the whole whiskey.
That's whiskey.
Yeah.
That's a whole other animal.
The whole stack is like not really ready for that.
Flask channels.
Yeah.
Yeah, exactly.
Like Django channels.
Yeah.
We need Daphne for, Daphne for Flask.
Right.
So maybe, I mean, maybe that's somewhere that needs some pushing.
I'm not actually familiar with all the options out there for that, but.
Yeah.
I don't know what that would look like.
That'd be interesting.
It's interesting to think about, but at the same time, it might not be a problem you have, right?
Like I find these web apps are actually quite fast already.
Yeah.
If you build things to be scalable, which just means they need to be predictably the
same speed.
Less stable.
And then you really don't have any problems.
And I think everything's pretty fine.
I don't think we have to really go into some bleeding, cutting edge concurrent worlds if we
don't have to.
I think it's wise for things that you're consuming, like for requests, for example,
we're planning on going to async.
You are?
Yeah.
I was about to ask you, what's the plans for requests?
So async is going to make its way in there.
So I could like await a request.get, maybe a get async?
Yeah, exactly.
There would be a new set of calls.
Like right now there's request.get, request.post.
So there would probably be a request at async.get or something like that.
And then you would await on that and it would just work.
That sounds like the business.
Yeah.
Are you familiar with, what is it, AIo HTTP clients?
I don't know exactly what.
I had Guido walk me through the code once and it seemed relatively straightforward.
It seemed like G event, but with explicit instead of the opposite.
Nice.
So I'm really excited to hear about this async request stuff.
Yeah.
We're working on it during the sprints.
So, well, Corey is working on it during sprints, I should say.
There's a project that we depend on called UR Lib 3.
And that is something that is being, V2 is being worked on by Corey, one of the core contributors
to requests.
Uh-huh.
And he is going to work on the preliminary async stuff.
He's going to get ready.
He's like refactoring it, rewriting it.
And it's going to be in preparation for adding async to that.
And then once we add async to that.
And then you're like, all right, we're going to do this.
And then we're going to do it.
Yeah.
And then in maybe six years, we might get to flip those APIs around.
So maybe the default in request would be asynchronous.
And then there would be async that would be separate.
Right.
But that would happen in like six years.
That really depends on how the community moves and stuff like that.
I honestly would be surprised if that's how things move.
But we'll see.
People find concurrency confusing.
They really do.
And I am hoping that I can really help bridge the gap with requests to make it be like,
this does not have to be scary.
This can be exactly what you're used to, except for magically amazing.
I'm really excited to hear that.
It is challenging.
But I was talking to somebody, listening to one of the other podcast Python Bytes that I do.
And I'd cover the AIOHCP client stuff.
And they said, I'm trying to remember the numbers.
It's like, oh, I had this project where we do a lot of scraping or API calls or something.
And it would take 16 minutes to run.
Yeah.
They flipped it to the async version.
And it ran in like 30 seconds.
And it actually crashed the machine because it was like doing too much.
They had to throttle it.
Yeah.
So I mean, just like.
It's because people don't understand.
I'm not saying they don't understand.
But I feel like there's a basic, simple computer science.
I'm not a computer science student by any means.
But there's like, you know, you have two things that a computer is limited by that your program is limited by.
It's IO bandwidth and CPU bandwidth.
And if you don't understand those things, it's very easy to.
If you do understand those things, you can not use these new async things.
And you can get very good performance with just using threads or just using multiprocess or in doing stuff like that.
The magic of async is that, you know, it just takes that to this logical extreme where you have these event loops that are just like.
Yeah.
And I think actually what they were going to, I think they might have run out of memory because they had too much pulled in at once.
Yeah.
Something like that.
But just the like.
It opens up a whole new can of worms of things that people don't understand.
And that's one of the things that I kind of prefer synchronous programming.
Yeah.
Myself.
Because it's something that is approachable and people understand.
And so I think I want to encourage people to understand how to better optimize synchronous code.
Yeah.
And there's some great things for that.
So there's a product like Minald's.
Have you seen that?
No, I don't think so.
It's a WSGI server that you can use with G Unicorn.
Okay.
And it uses an asynchronous event loop.
But you just turn it on.
And it just spawns every request.
Gets like a new green thread or something like that.
And your code just magically is asynchronous.
You know, and it's just like, that's how G Event works too.
But it's not using monkey patching.
Right.
And that's the level of async that I like is where it's at like the thread level.
Right.
I don't want it to be at the code level necessarily.
Unless it's something like requests doing it.
I don't personally want to be writing code that is dealing with this stuff.
I want to be consuming the code that's dealing with that stuff.
Right.
Yeah.
So I think those are solving slightly.
Like as a software engineer, I don't think people should have to touch the asyncio APIs.
I think that they should be consuming them.
Yeah.
And it's really interesting.
So I mean, on one hand, like any server environment, it's like almost by the time it gets to your code, it should already have taken care of that problem.
Precisely.
Yeah.
I mean, if you're on like your one.
Unless you're like working for a major company, you know, a company that has some major technical issues.
Not issues, but technical problems that they're challenges.
Yeah.
And they're like, we're going to solve this problem in this really cool way.
And then.
Your discuss, your Instagram.
Yeah.
Urban airship or something.
Exactly.
Yeah.
Precisely.
Exactly.
But most apps don't totally need it.
Yeah.
So, but that example of 16 minutes to 30 seconds really is like, wow, you could really get a lot out of that.
And to see if I could use requests to do that, that would just make me smile.
Yeah.
Well, you can do things like that today because there's a, there's a library called request futures, which is really great.
Okay.
And it uses the concurrency.futures module to allow you to spawn off either threads or sub processes or anything else.
There's also a G request, which I wrote a very long time ago, which is not very well maintained, but it allows you to use a G event with requests.
Okay.
So there are ways to do this today.
And again, you can do it at that module level.
So you just write your synchronous code and you just kind of wrap it in an asynchronous paradigm.
You know, use something like Celery and then you can, you, and then you get concurrency out the wazoo.
Awesome.
Yeah.
It's really cool.
Yeah.
This portion of Talk Python To Me is brought to you by Hired.
Hired is the platform for top Python developer jobs.
Create your profile and instantly get access to thousands of companies who will compete to work with you.
Take it from one of Hired's users who recently got a job and said,
I had my first offer within four days and I ended up getting eight offers in total.
I've worked with recruiters in the past, but they were pretty hit and miss.
I tried LinkedIn, but I found Hired to be the best.
I really liked knowing the salary upfront and privacy was also a huge seller for me.
Well, that sounds pretty awesome, doesn't it?
But wait until you hear about the signing bonus.
Everyone who accepts a job from Hired gets a $300 signing bonus.
And as Talk Python listeners, it gets even sweeter.
Use the link talkpython.fm/Hired and Hired will double the signing bonus to $600.
Opportunity is knocking.
Visit talkpython.fm/Hired and answer the door.
Let's maybe talk a little bit about PyCon.
We're sitting here.
Yes.
In an extremely large room that's actually luckily quite empty right now so we can get some good audio, but it's been very busy.
It has.
It's been very exciting.
What's your takeaway for the week?
You had a good time?
I've had a fantastic time so far.
I'm sad that you're speaking about it retrospectively.
It's in the past, right?
It's halfway over, I guess, ish, right?
Yeah, more or less.
And it breaks my heart because it's my favorite time of the year and I feel like I just got here.
And I did just get here.
Yeah, it was really amazing.
Every time I come here, it's just so many great people to meet.
And this year I had a booth, so I really got to meet more people.
I didn't have to randomly run into them.
Yeah, yeah.
You have the Heroku booth that you're hanging out at.
Yeah.
And it's just...
For me, PyCon is the one time of year that I have a family at home, of course, but it's really...
The Python community is my true family and I get to really be with my family.
And I get to do that when I...
I don't really go to as many conferences as I used to go to.
But PyCon has always been the home, like, coming home conference because you get to see almost every single person.
And they're my best friends in the world, you know?
And it's utterly fantastic.
It's awesome.
Yeah, there's a couple of people that I've been working with really closely for a long time but I've never physically seen.
My biggest takeaway from PyCon so far has been these fidget spinners.
The world has become fidgety.
They're solving a fidgeting problem this year.
Yeah, I went out to the Saturday market here in Portland and I just got a higher end one.
Because there was a booth here at PyCon that was giving out fidget spinners.
Tell people what the heck is this fidgety spinner thing.
It's something you hold in the center and then you spin it.
And then, like, you can rotate it and you feel a centrifugal force and it gives you something to do with your hands.
Yeah, I guess I'll try to describe it.
Like, imagine a little weighted circular thing with a fixed ball bearing like an axle almost in the middle.
Yeah.
You spin it really smoothly.
Yeah, they're kind of designed, at least the hype on the internet is saying that they're designed for kids with ADD.
Okay.
And I have ADD.
Yeah, okay.
And I find it to be very useful.
I usually am vaping like an electronic cigarette.
Yeah.
And that gives me something to do with my hands and I find it very pleasurable on many levels.
And I find it difficult to be a PyCon because I can't, like, be vaping when I'm walking around.
We're trapped inside, more or less, yeah.
Yeah, so I have, like, a nicotine patch on and that helps.
But when, as soon as I got one of these, I, like, totally forgot about smoking because it's just giving me something else to do.
So, it's, I'm actually really loving it.
Yeah.
Yeah, fidget spinners.
And I think it's going to help me a lot because I work from home and I'm in meetings and I'm usually smoking during the meetings and it's kind of rude.
So, I'm going to start, and during my podcast, if you listen to Import This, you'll hear, like, little, this weird puff sound.
It's because I'm vaping.
So, maybe I'll start fidgeting instead.
Yeah, you'll be fidgeting.
And those things are dead silent because they're, like, smooth ball bearings, so.
Yeah, yeah.
Do you want to try it?
Yeah, let me check this out.
Yeah.
All right.
Well, this is high-end.
Yeah, that one's really cool.
Wow.
It was $25, that one was.
This one was free.
And then this one was $15 I bought from a kid.
They set up a little shop downtown and the kids were selling them.
Kids don't do ice cream stands and lemonade stands?
No, they're selling spinners.
That's awesome.
They raise money for research for their autistic brother.
Wow, that is actually super cool.
Yeah, it was a great story.
I had to buy one.
I don't need three of them, but I, you know, I can't say no to that.
Yeah, that's right.
So, did you go check out the keynotes?
No, I do the hallway track at PyCon.
Yeah.
So, I make a point to not go to almost any talks.
I went to, Corey Benfield did a talk on requests under the hood and I went to support him at
his talk.
But that's the only talk I've been to so far.
And there's a really great talk at the end of the day tomorrow on NES video games and hacking
them with Python.
Yeah, that's got a lot of...
So, I'm probably going to try to make it to that one.
Yeah, I mean, people should check out, what is it, youtube.com slash PyCon 2017.
It's already, videos are already showing up there.
Oh, really?
Yeah, yeah.
There's like, I don't know about the talks yesterday, but the tutorials are up.
Yeah, and they have, they're transcribed live.
Yeah, it's really amazing.
These guys do a great job putting that on.
I'm really impressed with the accessibility of this conference.
I've seen people here with wheelchairs, which is, I don't, I've never noticed that before.
I'm sure it's been a thing.
But they also have teleprompters, not teleprompters, but what's the opposite of a teleprompter?
Telescribe?
Yeah, like a live transcriptionist type thing, whatever.
Yeah, like you have in court.
Yes, exactly.
But for the videos during the talks and even the questions that are being asked are being transcribed.
And I very, it's just, I love this conference.
It's a great conference.
Yeah.
And I go to a lot of conferences and PyCon is by far my favorite.
That's cool.
All right.
So you've been walking around, like I have been.
I'm not been going to, I went to the Jake Vander Plaats' keynote on why Python is popular
in science and it was actually brilliant.
I talked to him about that yesterday afterwards.
It's inspiring.
Was it really?
So the short takeaway is, like he said, like, look, Python is a mosaic of people using it
for different things and they're coming in from different angles.
The way, like, an astronomer might pull up some data and, like, play and iterate with
it is very different than, like, you and I might write a web app where we're going to
very structured.
Yeah, yeah.
And it's like a static thing that we build up over an architecture.
And just, like, talking to people from different backgrounds who use the same tools in different
ways gives you a really interesting perspective on life.
Oh, it does.
It does.
Yeah.
And Python gives us this common ground to build upon.
You can meet people who work on satellites and who work on web apps and who hack and who...
Yeah.
It's amazing.
And you're all, like, family to each other because you just speak the same language.
Yeah, it's amazing.
Last night I went to dinner with some of the guys from PyCharm and a bunch of people from
the Data for Democracy group.
A bunch of data scientists that are doing, like, volunteer work for democracy and freedom
around the world.
Yeah, it's just ridiculous.
I love this place.
I truly always regret not staying longer because I'm staying for only one day at the sprints.
Yeah.
And I always wish I'd stay longer because I really wish PyCon was all year round.
I just wish this was my life because it really is like coming home.
It's really nice.
Yeah, I totally recommend the...
If anyone is listening and hasn't come to PyCon, I want you to know that my first PyCon,
I paid for myself.
I didn't have an employer paying for me to go and I wasn't making much money at the time.
It was a very large expense, but they do have financial aid available.
I probably should have applied at the time.
It was worth every penny.
It totally changed my life.
And I had been doing Python for, like, quite a while at that point.
And it was a big deal for me to go to my first PyCon and I can't recommend it more.
That's awesome.
I definitely want to second that.
And as far as tech conferences go, this is pretty affordable for the size and scale.
Like, a lot of, like, OzCon.
Oh, yeah.
I looked at that.
It's like $1,500.
Conferences are extremely expensive.
This is a very approachable and doable conference.
It's like $300 or something.
Yeah, but when you add up the hotel costs and the flights and it's, you know, if you're not making, you know, a really good developer salary, then it's, you know, it's a lot of money.
If you can't drive in from, like, some four-hour radius, then it starts to really get intense.
Yeah.
Yeah, especially the hotel costs.
But they have financial aid available, too.
So if there's any reason that finances are a reason that you can't come to PyCon, apply.
And they'll, like, give you a free ticket.
It would be cool for somebody to set up some kind of, like, web app before these events.
So people who are, like, looking to room share and, like, save money.
Yeah, yeah, yeah.
They could, like, kind of be, like, find your PyCon roommate.
Forgot about that.
Wouldn't that be fun?
That'd be a fun project to set up.
Wouldn't that be a fun project?
Well, I know in years past that was a thing.
Okay.
Because I remember the year I went myself, there was, like, a mailing list of people trying to find room shares.
I'm not sure.
It must still be a thing.
Yeah, probably.
I assume.
But maybe not.
I don't know.
It'd be cool to make something like that.
I assure you it did exist.
Yeah.
Awesome.
All right.
So, yeah, definitely if you guys are out there listening and you haven't come to PyCon, you should come.
But you should get your tickets straight away or you won't be able to come.
Yeah, it sells out.
And it's cheaper the earlier you buy them, too.
Absolutely.
Yeah.
All right.
Kenneth, thanks for being on the show, man.
Always a pleasure.
I'm a huge fan of the show.
And I want to really thank you for all the great advice you give all these developers.
And I listen to you when I'm taking baths all the time.
Oh, thank you so much.
That is awesome.
You're the only one I still listen to.
I don't listen to podcasts often enough to be caught up on them, but you're always the one I pull up to listen to.
Wow.
So, I listen to you on the way here, actually.
How cool.
Awesome.
Yeah.
So, really quick before we go, tell people about your podcast.
Oh, I have a Python podcast.
It's called Import This.
If you Google Import This, Kenneth writes, you'll find it.
It's on SoundCloud.
And I do very long interviews with Python friends of mine.
And we just catch up and talk about Python.
And actually, we talk about things that are unrelated to Python often, like their lives and stuff like that.
And I just put it.
It's very unfiltered, uncandid.
It's a raw, natural conversation.
Yeah.
There's no editing or anything like that.
I mean, it's produced, but it's not.
I just hit record, basically.
Awesome.
So, it's a great little thing.
And some people really like the longer format podcasts.
So, it's one of those.
If you need something on your commute, you should check it out.
Awesome.
And I'll definitely be sure to link to it in the show notes of people and just click on it and go there.
Yeah, it'd be awesome.
All right.
Thanks again for being on the show.
Yeah, definitely.
This has been another episode of Talk Python To Me.
Today's guest has been Kenneth Wright.
And this episode has been brought to you by CodeShip and Hired.
Do you have software?
Would you like to know if it has bugs before you deploy it?
Then jump over to talkpython.fm/code ship and set up a free CodeShip basic account,
ship tested software.
Hired wants to help you find your next big thing.
Visit talkpython.fm/hired to get five or more offers with salary and equity presented
right up front and a special listener signing bonus of $600.
Are you or your colleagues trying to learn Python?
Well, be sure to visit training.talkpython.fm.
We now have year-long course bundles and a couple of new classes released just this week.
Have a look around.
I'm sure you'll find a class you'll enjoy.
Be sure to subscribe to the show.
Open your favorite podcatcher and search for Python.
We should be right at the top.
You can also find the iTunes feed at /itunes, Google Play feed at /play, and direct
RSS feed at /rss on talkpython.fm.
Our theme music is Developers, Developers, Developers by Corey Smith, who goes by Smix.
Corey just recently started selling his tracks on iTunes, so I recommend you check it out at
talkpython.fm/music.
You can browse his tracks he has for sale on iTunes and listen to the full-length version
of the theme song.
This is your host, Michael Kennedy.
Thanks so much for listening.
I really appreciate it.
Smix, let's get out of here.
Outro Music.
*outro music*
Thank you.