# 22 Sep 2019, 02:03PM: Futureproofing Your Python Tools:
The people who maintain Python and key Python platforms want to help you protect the code you write and depend on.
If you write software in Python, or depend on something that's in Python, this is for you.
Some of you are writing Python 2, or you still have software you depend on that is written in Python 2. January 1st, 2020, is the day that official support for Python 2 stops. So this is a fresh heads-up that you should really have a migration plan and start working on it, to move to Python 3. A lot of stuff you depend on already works on Python 3, and is even pledging to end Python 2 support in or before 2020. And it's easier than it's ever been to port your own code from 2 to 3.
You should probably upgrade to Python 3.7. If you want to test out 3.8, it has some changes in how it does warnings, and the first release candidate is out.
But, speaking of futureproofing:
Code authors move in and out of projects and companies. Six months or 18 months later, maybe you want to update and re-use, re-release or re-deploy code someone else wrote. Or you want your team to be able to reuse what you did after you leave, which means you need the code to run, and you need them to have the password so they can update stuff.
Have you written Python code that they have published as a package on the Python Package Index, pypi.org, so other people can use pip install to install it?
(And if you want to do that but don't know how? Check out this recently improved tutorial to help you do that.)
Publishing that package is a great way of making it so other people can run and deploy it, even within other parts of your organization.
But -- who actually has the keys to the castle? Who can upload a new version, or delete a version that has a problem?
You should probably make sure multiple people have either "owner" or "maintainer" privileges on the project on PyPI.
And you should review your project security history display, which lists sensitive events (such as "file removed from release version 1.0.1") in your PyPI user account and your PyPI project. We just added this display, so you can look at things that have happened in your user account or project, and check for signs someone's stolen your credentials.
And then how do you make it a little harder for vandals, spammers, and thieves to take over your account and upload malware or delete all the packages? Add two-factor auth for your login, like you would with your bank. Use an app on your phone to give you a six-digit code, or use a physical security device like a Yubikey.
And how do you make it easier to automate publishing new versions of your package, and make it safer to save your credential in the cloud? We've made it possible for you to create an "API token" where all it can do is upload, so you can use that instead of your PyPI password.
With well-tested Python 3 migration tools and new PyPI security features, now is a great moment to invest in robustness for Python software that you make or depend on.
[This blog post is kind of unwieldy, because it's about too many different things. I won't be publicizing it that much and instead will probably reuse text from it in more focused announcements elsewhere. But I'm publishing it here as a summary of my recent work, because management and communications for the projects above are what I've been working on recently for the Python Software Foundation. (A different kind of summary is on the Clients page for Changeset Consulting.)]
# (1) 08 Aug 2019, 11:49AM: When/How Do People Decide To Apply To CS Grad School?:
I'm trying to understand how people decide to go to CS grad school in the US. For instance, what proportion of PhD applicants are coming straight from undergrad, versus another graduate degree (such as an MS in math), versus industry? Do they generally decide first on what they want to research, whom they want to work with, or that they want to go to grad school at all? I presume there is a survey of this somewhere?
I've found the Computing Research Association's Center for Evaluating the Research Pipeline's report on why grad students choose computing, and I've also looked at the National Center for Science and Engineering Statistics website. I have found a little data on what proportion of doctoral recipients were previously in a baccalaureate program, a master's program, or industry, but not about what proportion of applicants, or
accepted applicants, come from those categories.
(Maybe there's paywalled research on this within the ACM's special interest group on CS education? If so, lemme know and I will try looking for that?)
The reason I am asking this is to help professors and guidance counselors I know (maybe you've heard that Outreachy has a new career advisor). They want to improve their abilities to help students and programmers consider research careers, and better target their outreach consider applying to specific graduate programs. How does the engagement funnel currently work? And thus, where are the gaps? I presume there are a bunch of people who would do well in grad school, and find it fulfilling, and use their research and their degrees in ways that would benefit the world, but no one ever says to them "hey have you thought about going to grad school," so they don't think of it as a possible thing for them.
# 05 Aug 2019, 09:01AM: Kickoff for Communications Work on the Python 2 Sunsetting:
Python's 2.x line will reach End of Life on January 1, 2020, meaning that the maintainers of Python 2 will stop supporting it, even for security patches. Many institutions and codebases have not yet ported their code from Python 2 to Python 3. And many of them haven't even heard yet about the upcoming EOL. Volunteers have made many resources to help publicize and educate, but there's still more work to be done.
So the Python Software Foundation has contracted with my firm, Changeset Consulting, to help communicate about the sunsetting of Python 2. The high-level goal for Changeset's involvement is to help users through the end of the transition, help with communication so volunteers are not overwhelmed, and help update public-facing assets so core developers are not overwhelmed.
During this project (starting now and going till the end of April 2020), Changeset's goals are:
- Create a concise page on python.org giving community guidance on the why and what of EOL
- Create and publicize a formal press release through the PSF
- Audit and update every mention of Python 2 on python.org, including in documentation and the wiki
- Develop a plan to handle redirecting https://docs.python.org/2/* (especially deep links)
- Publicize the new "what's up with EOL" page to a variety of audiences, respond to questions, and keep the page updated until PyCon US 2020 based on questions that come up
So, towards those goals, you'll see me with my colleagues:
- Researching, writing, and editing technical documents and the press release
- Corresponding with stakeholders, writing and publishing announcements and other communications in multiple media
- Initiating and facilitating meetings (I will try to make them very minimal and short; if I invite you, please let me know your response)
- Filing, triaging, labelling, prioritizing, and linking issues, such as in the website repo
For accountability, Changeset will provide reporting via:
I'm going to be asking for a lot of help along the way from the Python community: meeting with us, answering our questions, double-checking our drafts for accuracy, publicizing that EOL page to your circles, setting up some parties for January 1st. Thanks in advance and let's get the Python user base further along towards enjoying the shininess of Python 3.
# 01 Aug 2019, 09:09AM: Background Music:
So in my household we have a zillion little shared references, and some of those are about pop songs of the late 20th century. For instance, if we're in a restaurant or something and we hear "Higher Love" by Steve Winwood (I just had to look that up, it's not like I knew the name of the song or the artist already), we laugh because of the time Leonard pointed out that the main lyric kinda sounds like a complaint a customer might give a server.
Bring me a higher love
This love is insufficiently high
Leave bad review on Yelp
(Upon a full listen: the synth riff from 3:04 to 3:11 reminds me of the start of the Doogie Howser, M.D. opening theme. A lot of the folks I meet are not people who went to schools in the US in the late 1980s/early 1990s while younger than approximately everyone else in their grade cohort, and thus they did not experience being called "Doogie". Nor did they experience Head of the Class which was -- for me -- sympathetic representation of book-smart nerddom in mass media. Not sure I'd feel that way if I re-watched it now.)
Every once in a while we go use YouTube to watch the music videos for songs that are in sort of the "you will hear these in public spaces in the US" canon but that we've never really listened to. Always feels like popping the hood in a car where up till now I've just been a passenger.
# 30 Jul 2019, 01:18PM: Comparing Y2K and Climate Change:
When you know there's a big upcoming threat, how do you get big institutions to commit and follow through? And in particular, how useful is it to frighten whole populaces?
I looked into a specific claim on this topic today because a MetaFilter member compared climate change to the Y2K bug, and said,
from my POV as a computer geek who saw what it took to get the... very fine people... in management moving was panic and doom saying....
...those calm and measured meetings only took place because of the general panic. If they were going to have calm rational meetings without the outside pressure of people convinced their toasters would explode (or whatever) they'd have done it earlier.
From my POV panicking the masses, while not good, was literally the only reason the billionaires paid the slightest attention to the problem. The billionaires and the rest of the management types didn't panic, or not much anyway, but their non-panic reaction was produced entirely by the panic.
So I wanted to double-check whether that is in fact the dynamic that actually happened back in the 1980s and 1990s. Was the causality fairly simple, or was the course of events was shaped by lots of conversations among regulators, shareholders, customers, boards of directors, etc. that weren't as visible to a working engineer's point of view?
So I started looking for a little more research and data about Y2K mitigation/prevention decisionmaking especially on the institutional level, beyond the US Senate special committee report. This question -- what caused institutions to take Y2K seriously? -- seemed like the kind of thing historians of technology and organizational sociologists and political scientists must have studied. I did some digging and here are some things I think are useful to consider:
- Y2K was a technology problem at heart, and so IT investment, where IT sat in an organization, senior leaders' attitudes towards IT, etc. were factors that slowed down planning & compliance.
- Banks, for instance, found it hard to get their customers/users to understand the importance of Y2K compliance, because it was an abstruse technological issue.
- Getting companies to actually act involved some amount of influence/pressure from the outside, especially from the mass media and from regulators, state and federal agencies, and industry-wide consortia and working groups (including credible experts talking about the risk of legal liability), but also people inside companies needed to believe that the threat was not being overstated by doomsayers or IT departments seeking more turf.  
- Auditors at private companies -- whose work was often required by insurers -- did lots of disclosure, separately from anything customers or the public at large demanded via grassroots action.
- Y2K was a software problem, and so countries that made a lot of stuff but made and used way less software didn't need to do nearly as much to address it. I see a bunch of stuff in the Y2K prep literature about the US and the UK, and way less about, for instance, China.
- Companies in countries with stronger political rights and civil liberties disclosed more about their Y2K preparedness. I'd assume that disclosure made market pressure more possible (from investors and customers) but I haven't researched it.
Apologies for how much of that research is behind paywalls.
I only looked at this for a few hours but (a) I think there are a lot of important differences between the structure of the Y2K problem (and its mitigations) and climate change, and (b) I believe the story's a lot more complicated than "mass panic -> billionaires finally paid up". I think widescale media attention to the concern played a big part in directly motivating regulators and institutional decisionmakers, and grassroots concern in the general public (as citizenry and as customer base) played a part in that, but it looks like insurers and auditors and industry groups were also really crucial. To sum up, yeah, we have lessons to learn from Y2K -- I'm drawn to Bob Bennett's phrasing, that we must be Paul Revere but not Chicken Little -- but I'm very hesitant about drawing the conclusion that literally panicking the world's populaces is the only way to drive climate change mitigation.
 Ho, A. T.-K., & Smith, J. F. (2001). Information Technology Planning and the Y2K Problem in Local Governments. The American Review of Public Administration, 31(2), 158–180. https://doi.org/10.1177/02750740122064901
 Huang, J. C., Newell, S., & S-L, P. (2001). The process of global knowledge integration: A case study of a multinational investment bank's Y2K program. European Journal of Information Systems, 10(3), 161-174. doi:http://dx.doi.org/10.1057/palgrave.ejis.3000402
Backus, George, et al. "Comparing Expectations to Actual Events: The Post Mortem of a Y2K Analysis." System Dynamics Review, vol. 17, no. 3, 2001, pp. 217. doi:http://dx.doi.org/10.1002/sdr.217.
 Chang, Hsiu-Hua, Chun-Po Yin, and Huey-Wen Chou. "Diffusion of Enterprise Resource Planning Systems in Taiwan: Influence Sources and the Y2K Effect." International Journal of Enterprise Information Systems, vol. 4, no. 1, 2008, pp. 34-47. doi:http://dx.doi.org/10.4018/jeis.2008010103.
 Solomon, H. (2005, Sep 23). Y2K: The disaster that wasn't. Computing Canada, 31, 46-48.
 Clarkson, Peter M., Colin Ferguson, and Jason Hall. "Auditor Conservatism and Voluntary Disclosure: Evidence from the Year 2000 Systems Issue." Accounting and Finance, vol. 43, no. 1, 2003, pp. 21.
 S, M. W. (2004). An international investigation of associations between societal variables and the amount of disclosure on information technology and communications problems: The case of Y2K. The International Journal of Accounting, 39(1), 71-92.
# 22 Jul 2019, 11:35AM: Beautiful Soup is on Tidelift:
I've been doing a tiny bit of consulting for Tidelift for a little over a year now, mainly talking about them to open source maintainers in the Python world and vice versa. (See my October 2018 piece "Tidelift Is Paying Maintainers And, Potentially, Fixing the Economics of an Industry".) And lo, in my household, my spouse Leonard Richardson has signed up as a lifter for Beautiful Soup, his library that helps you with screen-scraping projects.
There was a period of about a year in 2017-2018 when I wasn't interested in doing Beautiful Soup work, but Tidelift changed that. Tidelift gathers subscription money from companies that rely on free software, and distributes the money to the developers in exchange for a level of support that I find sustainable.
Nobody builds an entire product around Beautiful Soup (or at least nobody will admit do doing this), but thousands of people have used Beautiful Soup to save time at their day jobs. Bundling Beautiful Soup together with bigger projects like Flask and numpy is a solution that works really well for me.
The other day I looked at the list of featured supported packages and was happy to see that a bunch of Python projects have signed up as lifters, including SciPy and numpy, Flask, setuptools, Werkzeug, websockets, urllib3, celery, coverage, a bunch of Django packages, Jinja2, keyring, Pylint, coverage, and pytest. And I think over the next 6-12 months we're going to see some effects of Tidelift support -- not just in the security and release cadences of the supported projects, but on other issues stemming from unfairness and a lack of reciprocity in open source, like maintainer burnout and expectation-setting. The list of licensing, security, maintenance, and marketing tasks lifters agree to do may end up being a benchmark, like the open source Independent Verification & Validation checklist by Open Tech Strategies, that even non-lifter maintainers use to set realistic expectations for what "supported"/"maintained" means.
# 12 Jun 2019, 10:43AM: Hey, You Left Something Out:
Of course not all the responses I get to my work are positive. Sometimes I get criticism. And a subset of that criticism says more about the person giving it than about the quality of what I've made. I try to keep a thick skin about that but I don't always succeed.
One particular kind of response has piqued my interest lately. Some of the feedback I get means to be praise, but contains a kinda-joking complaint about something that the person thinks I left out. I saw this recently in a recommendation of my PyCon 2016 talk, "HTTP Can Do That?!", and in another commenter's response. And some commentary of the "they/you left x out" variety is straightforward criticism.
At its most loving, I think this kind of commentary means to be a kind of "yes-and" response, sharing the experience of enjoying something and extending it by recommending another related thing. (I have been working on this blog post, on and off, for a few months; the day I am posting it, I see a perfect example.) And I can empathize with that!
But, a lot of the time, this kind of response comes with an explicit marker or implicit connotation of complaint: the author/speaker did not mention the thing that I think should be mentioned, and therefore, something is wrong.* Perhaps a more useful approach would be to wonder, in a genuinely curious way, why the author didn't mention it? Was it out of ignorance? Was it a deliberate choice, and, if so, to what purpose?
Marco Rogers's recently observed: "A lot of men seem to have been conditioned to think that telling someone that you disagree is the same as asking them a question. Like the way they learn to engage is by *creating a conflict*." Maybe that plays into this.
And as Josh Millard notes,
There's a lot of this sort of detached entitlement out there.... "I want content generated to my tastes" collides with "I'm making something with my bare hands" in such a way that the folks in the more passive former camp feel somehow totally comfortable asserting the high ground on the people in the latter.
Personal taste is personal taste and everybody's got a right to it; criticism is useful, at least when it's useful. Beyond that, though, there's a lot of Why Am I Not Being Correctly Entertained out there in the world that manages to get off the leash for no good reason, and from the doing-the-work, learning-the-craft, making-the-content side of things that does get awful tiring.
And maybe that plays into this too.
Compilation-makers, list-makers, etc. run into this kind of criticism frequently, as fanvidders discussed in a Vividcon panel about multisource vids. Perhaps some readers read any list of things sharing particular characteristic as an attempt to make the one canonical list, and thus read any publicly shared list as implicitly inviting corrections and additions toward this goal.
Last year bironic commented wryly,
I love how many multifandom vids lately come with explainers about scope, as we brace for people to come in and yell about someone who was included or left out.
And I appreciate vidder thingswithwings's response:
...there are so many selection choices to make, and only so many seconds of song . . . I think it's good to make it clear that we're making these decisions thoughtfully...
That's the spirit I see in thingswithwings's vidder's notes on their joyous, spirited and dancey vid "Gettin' Bi" and eruthros's vidder's notes on her excellent, moving, incisive vid "Straightening Up the House". And that's the spirit I'd like to inhabit as I make and share recommendation lists, compilations, etc. going forward.
And in that spirit I'll address here the praise-complaints of my own work that I linked to in my second paragraph. I scoped "HTTP Can Do That?!" to discuss underappreciated real, working parts of HTTP and share examples of things that work, even if they're bad ideas, as illustrations. I didn't show the cover of Bradbury's Fahrenheit 451 in my talk because -- as I mentioned during the Q&A -- I think it's fine to leave that particular connection as a bit of an Easter egg so some people have something to figure out when they look up response status code 451 later. I didn't include the teapot response code (418) because it's already fairly popular and well-known as a joke response code, and I wanted to spend my time on stuff folks weren't as likely to run across in other fora, and because it's a joke that isn't in the HTTP standards. I made a tradeoff between concision and nuance. Similarly, I didn't use the word "neoliberal" in that post about feelings of overwhelmption because that wasn't the point.
People who want to compliment work should probably learn to give compliments that sound encouraging. As one writer notes: "I think Twitter, for all its good qualities, can very much be a Killer Of Work exactly because people don't know how to say "that's so awesome!" or lift creators up in the idea stage." And people who genuinely want to submit you-left-something-out bug reports about someone else's work** should probably spend a few moments checking the maker's stated criteria and purpose, and reflecting on whether they perhaps had an interesting reason for the exclusion or omission, or on how much the gut biomes of the creator's intended audience matches the reader's gut biome. "I'm curious about the choice you made" may sound passive-aggressive, but I'd rather hear that than something that's just flat-out aggressive.
(Oh, and to be tiresomely empowering again: a human created the thing you're responding to; you're a human and you could make a thing, too.)
* "You forgot Poland" always comes to mind, even though a face-to-face debate is such an unusual context compared to the ways I usually get feedback like "you forgot x".
** even something tiny like a single joke
Thanks to Mindy Preston and others who commented on drafts of this idea & piece.