Categories: sumana | Open Source and Free Culture
Free and open source software and free culture communities, norms, projects, and experiences
# 26 Sep 2019, 07:36AM: Cambrian explosion of discussions about the future of software freedom:
In 2009, Dreamwidth user rydra-wong did the great favor of making link roundups to help people keep track of a distributed conversation happening on lots of people's blogs about a current controversy.
I'd love for someone to take up that kind of work ("linkspam" roundups) or point to something similar, a Pinboard tag or a TagTeam instance/tag/team, to help us all see people writing about the future of open source software licensing. This post by Molly de Blanc and this one by Karl Fogel, for instance.
# 22 Sep 2019, 02:03PM: Futureproofing Your Python Tools:
The people who maintain Python and key Python platforms want to help you protect the code you write and depend on.
If you write software in Python, or depend on something that's in Python, this is for you.
Some of you are writing Python 2, or you still have software you depend on that is written in Python 2. January 1st, 2020, is the day that official support for Python 2 stops. So this is a fresh heads-up that you should really have a migration plan and start working on it, to move to Python 3. A lot of stuff you depend on already works on Python 3, and is even pledging to end Python 2 support in or before 2020. And it's easier than it's ever been to port your own code from 2 to 3.
You should probably upgrade to Python 3.7. If you want to test out 3.8, it has some changes in how it does warnings, and the first release candidate is out.
But, speaking of futureproofing:
Code authors move in and out of projects and companies. Six months or 18 months later, maybe you want to update and re-use, re-release or re-deploy code someone else wrote. Or you want your team to be able to reuse what you did after you leave, which means you need the code to run, and you need them to have the password so they can update stuff.
Have you written Python code that they have published as a package on the Python Package Index, pypi.org, so other people can use pip install to install it?
(And if you want to do that but don't know how? Check out this recently improved tutorial to help you do that.)
Publishing that package is a great way of making it so other people can run and deploy it, even within other parts of your organization.
But -- who actually has the keys to the castle? Who can upload a new version, or delete a version that has a problem?
You should probably make sure multiple people have either "owner" or "maintainer" privileges on the project on PyPI.
And you should review your project security history display, which lists sensitive events (such as "file removed from release version 1.0.1") in your PyPI user account and your PyPI project. We just added this display, so you can look at things that have happened in your user account or project, and check for signs someone's stolen your credentials.
And then how do you make it a little harder for vandals, spammers, and thieves to take over your account and upload malware or delete all the packages? Add two-factor auth for your login, like you would with your bank. Use an app on your phone to give you a six-digit code, or use a physical security device like a Yubikey.
And how do you make it easier to automate publishing new versions of your package, and make it safer to save your credential in the cloud? We've made it possible for you to create an "API token" where all it can do is upload, so you can use that instead of your PyPI password.
With well-tested Python 3 migration tools and new PyPI security features, now is a great moment to invest in robustness for Python software that you make or depend on.
[This blog post is kind of unwieldy, because it's about too many different things. I won't be publicizing it that much and instead will probably reuse text from it in more focused announcements elsewhere. But I'm publishing it here as a summary of my recent work, because management and communications for the projects above are what I've been working on recently for the Python Software Foundation. (A different kind of summary is on the Clients page for Changeset Consulting.)]
# 05 Aug 2019, 09:01AM: Kickoff for Communications Work on the Python 2 Sunsetting:
Python's 2.x line will reach End of Life on January 1, 2020, meaning that the maintainers of Python 2 will stop supporting it, even for security patches. Many institutions and codebases have not yet ported their code from Python 2 to Python 3. And many of them haven't even heard yet about the upcoming EOL. Volunteers have made many resources to help publicize and educate, but there's still more work to be done.
So the Python Software Foundation has contracted with my firm, Changeset Consulting, to help communicate about the sunsetting of Python 2. The high-level goal for Changeset's involvement is to help users through the end of the transition, help with communication so volunteers are not overwhelmed, and help update public-facing assets so core developers are not overwhelmed.
During this project (starting now and going till the end of April 2020), Changeset's goals are:
- Create a concise page on python.org giving community guidance on the why and what of EOL
- Create and publicize a formal press release through the PSF
- Audit and update every mention of Python 2 on python.org, including in documentation and the wiki
- Develop a plan to handle redirecting https://docs.python.org/2/* (especially deep links)
- Publicize the new "what's up with EOL" page to a variety of audiences, respond to questions, and keep the page updated until PyCon US 2020 based on questions that come up
So, towards those goals, you'll see me with my colleagues:
- Researching, writing, and editing technical documents and the press release
- Corresponding with stakeholders, writing and publishing announcements and other communications in multiple media
- Initiating and facilitating meetings (I will try to make them very minimal and short; if I invite you, please let me know your response)
- Filing, triaging, labelling, prioritizing, and linking issues, such as in the website repo
For accountability, Changeset will provide reporting via:
I'm going to be asking for a lot of help along the way from the Python community: meeting with us, answering our questions, double-checking our drafts for accuracy, publicizing that EOL page to your circles, setting up some parties for January 1st. Thanks in advance and let's get the Python user base further along towards enjoying the shininess of Python 3.
# 22 Jul 2019, 11:35AM: Beautiful Soup is on Tidelift:
I've been doing a tiny bit of consulting for Tidelift for a little over a year now, mainly talking about them to open source maintainers in the Python world and vice versa. (See my October 2018 piece "Tidelift Is Paying Maintainers And, Potentially, Fixing the Economics of an Industry".) And lo, in my household, my spouse Leonard Richardson has signed up as a lifter for Beautiful Soup, his library that helps you with screen-scraping projects.
There was a period of about a year in 2017-2018 when I wasn't interested in doing Beautiful Soup work, but Tidelift changed that. Tidelift gathers subscription money from companies that rely on free software, and distributes the money to the developers in exchange for a level of support that I find sustainable.
Nobody builds an entire product around Beautiful Soup (or at least nobody will admit do doing this), but thousands of people have used Beautiful Soup to save time at their day jobs. Bundling Beautiful Soup together with bigger projects like Flask and numpy is a solution that works really well for me.
The other day I looked at the list of featured supported packages and was happy to see that a bunch of Python projects have signed up as lifters, including SciPy and numpy, Flask, setuptools, Werkzeug, websockets, urllib3, celery, coverage, a bunch of Django packages, Jinja2, keyring, Pylint, coverage, and pytest. And I think over the next 6-12 months we're going to see some effects of Tidelift support -- not just in the security and release cadences of the supported projects, but on other issues stemming from unfairness and a lack of reciprocity in open source, like maintainer burnout and expectation-setting. The list of licensing, security, maintenance, and marketing tasks lifters agree to do may end up being a benchmark, like the open source Independent Verification & Validation checklist by Open Tech Strategies, that even non-lifter maintainers use to set realistic expectations for what "supported"/"maintained" means.
# 22 Apr 2019, 04:41PM: Rabbit Hole Interview(s):
Recently, the Rabbit Hole developers' podcast interviewed me; we discussed open source sustainability, maintainership, sensationalism among bards who sang the Odyssey, how PyPI is like Wikipedia, and what we think is paranoid.
The interview continued into a second episode discussing PyCon and The Art of Python, my past talks and plays, Halt and Catch Fire, what conferences are for, and the feeling of giving a bad talk.
Thanks to Stride for providing rough transcripts along with the audio!
A listener punned on my username ("brainwane") to tell me, "loved your perspective and insight on the podcast ... for me, it was 'braingain'". Awww!
We recorded these episodes on 27 February. The 7:17-08:06 segment of the first one proved prescient:
David:... NPM does an audit of the packages and says, okay, like, "this version is flagged with a known vulnerability, you should upgrade this." And it will just hammer you with that [unintelligible], infinitely, until you handle it. But like, you know, that’s also a form of open source software, that we’re depending on to nudge us.
Sumana: Right, and then the question of, again, sustainability, of like, well, is NPM, as a venture-backed thing, right..... You stay in this industry long enough and VC sounds like a dangerous term for anything you’re actually going to depend on.
David: Yeah, like the idea of something like PyPI going away. Like, I don't know what I would do? I would just have to find all of the binaries on a website? And like host my own... thing? Or...?
Stride released this episode on 19 March. On 22 March, surprising staff and at least this observer, npm laid off a number of workers on its open source team.
Please note that you can make a one-time or recurring donation of any amount to the Python Software Foundation that specifically supports PyPI and related packaging and distribution work (disclaimer: the PSF currently pays Changeset Consulting to work on PyPI and packaging), and that your org can sponsor the PSF for as little as USD$500 per year. And I am, as always, speaking here entirely for myself and not for any of my clients or colleagues.
# 19 Apr 2019, 11:00AM: Design, and Friction Preventing Design Improvement, in Open Tech:
This month a Recurser I know, Pepijn de Vos, observed a concentration of high-quality open source software in the developer tools category, to the exclusion of other categories. With a few exceptions.
I understood where he's coming from, though my assessment differs. I started reflecting on those exceptions. Do they "prove the rule" in the colloquial sense that "every rule has exceptions," or do they "prove the rule" in the older sense, in that they give us an opportunity to test the rule? A few years ago I learned about this technique called "appreciative inquiry" which says: look at the unusual examples of things that are working well, and try to figure out how they've gotten where they are, so we can try to replicate it. So I think it's worth thinking a bit more about those exceptional FLOSS projects that aren't developer tools and that are pretty high-quality, in user experience design and robust functionality. And it's worth discussing problems and approaches in product management and user experience design in open source, and pointing to people already working on it.
FLOSS with good design and robust functionality: My list would include Firefox, Chromium, NetHack, Android, Audacity, Inkscape, VLC, the Archive Of Our Own, Written? Kitten!, Signal, Zulip, Thunderbird, and many of the built-in applications on the Linux desktop. I don't have much experience with Blender or Krita, but I believe they belong here too. (Another category worth thinking about: FLOSS software that has no commercial competitor, or whose commercial competitors are much worse, because for-profit companies would be far warier of liability or other legal issues surrounding the project. Examples: youtube-dl, Firefox Send, VLC again, and probably some security/privacy stuff I don't know much about.)
And as I start thinking about what helped these projects get where they are, I reach for the archetypes at play. I'll ask James and Karl to check my homework, but as I understand it:
Mass Market: NetHack, VLC, Firefox, Audacity, Inkscape, Thunderbird, youtube-dl
Controlled Ecosystem: Zulip, Archive Of Our Own
Business-to-business open source: Android, Chromium
Rocket Ship To Mars: Signal
Bathwater? Wide Open? Trusted Vendor? not sure: Written? Kitten!
The only "Wide Open" example that easily comes to mind for me is robotfindskitten, a game which -- like Written? Kitten! -- does one reasonably simple thing and does it well. Leonard reflected on reasons for its success at Roguelike Celebration 2017 (video). But I'd be open to correction, especially by people who are familiar with NetHack, VLC, Audacity, Inkscape, or youtube-dl development processes.
Design: Part of de Vos's point is about cost and quality in general. But I believe part of what he's getting at is design. Which FLOSS outside of developer tooling has good design?
In my own history as an open source contributor and leader, I've worked some on developer tools like PyPI and a linter for OpenNews, but quite a lot more on tools for other audiences, like MediaWiki, HTTPS Everywhere, Mailman, Zulip, bits of GNOME, AltLaw, and the WisCon app. The first open source project I ever contributed to, twelve years ago, was Miro, a video player and podcatcher. And these projects had all sorts of governance/funding structures: completely volunteer-run with and without any formal home, nonprofit with and without grants, academic, for-profit within consultancies and product companies.
So I know some of the dynamics that affect user experience in FLOSS for general audiences (often negatively), and discussed some of them in my code4lib keynote "User Experience is a Social Justice Issue" a few years ago. I'm certainly not alone; Simply Secure, Open Source Design, Cris Beasley, The Land, Clar, and Risker are just a few of the thinkers and practitioners who have shared useful thoughts on these problems.
In 2014, I wrote a few things about this issue, mostly in public, like the code4lib keynote and this April Fool's joke:
It turns out you can go into your
Wikimedia and pushback: But I also wrote a private email that year that I'll reproduce below. I wrote it about design change friction in Wikimedia communities, so it shorthands some references to, for instance, a proposed opt-in Wikimedia feature to help users hide some controversial images. But I hope it still provides some use even if you don't know that history.
init.cfg file and change the usability flag from 0 to 1, and that improves user experience tremendously. I wonder why distributions ship it turned off by default?
I wanted to quickly summarize some thoughts and expand on the
conversation you and I had several days ago, on reasons Wikimedia
community members have a tough time with even opt-in or opt-out design
changes like the image filter or VisualEditor or Media Viewer.
- ideology of a free market of ideas -- the cure for bad speech is more
speech, if you can't take the heat then you should not be here, aversion
to American prudishness etc., etc. (more relevant for image filter)
- relatedly "if you can't deal with the way things are then you are too
stupid to be here" (more applicable to design simplifications like Media
Viewer and VisualEditor)
- people are bad at seeing that the situation that has incrementally
changed around them is now a bad one (frog in pot of boiling water); see
checkbox proliferation and baroque wikitext/template metastasis
- most non-designers are bad at design thinking (at assessing a design,
imagining it as a changeable prototype, thinking beyond their initial
personal and aesthetic reaction, sussing out workflows and needs and
assessing whether a proposed design would suit them, thinking from other
people's points of view, thinking from the POV of a newcomer, etc.)
- relatedly, we do not share a design vocabulary of concepts, nor
principles that we aim to uphold or judge our work against (in contrast
see our vocabulary of concepts and principles for Wikipedia content,
e.g. NPOV, deletionism/inclusionism)
- so people can only speak from their own personal aesthetics and
initial reactions, which are often negative because in general people
are averse to surprise novelty in environments they consider home, and
the discourse can't rise beyond "I don't like it, therefore it sucks"
- past history of difficult conversations, sometimes badly managed (e.g.
image filter) and too-early rollout of buggy feature as a default (e.g.
VisualEditor), causes once-burned-twice-shy wariness about new WMF features
- Wikimedians' core ethos: "It's a wiki" (if you see a problem, e.g. an
error in a Wikipedia article, try to fix it); everyone is responsible
for maintaining and improving the project, preventing harm
- ergo people who feel responsible for the quality of the project are
like William F. Buckley's "National Review" in terms of their
conservatism, standing athwart history yelling "stop"
I haven't answered some questions: what are the common patterns in our success stories (governance, funding, community size, maintainership history, etc.)? How do we address or prevent problems like the ones I mentioned seeing within Wikimedia? But it's great to see progress on those questions from organizations like Wikimedia and Simply Secure and Open Tech Strategies (disclosure: I often do work with the latter), and I do see hope for plausible ways forward.
# 27 Feb 2019, 09:01AM: GSoC/Outreachy Mentoring Orgs: Consider Giving Applicants English Tutoring:
Google Summer of Code just announced the 207 mentoring organizations (open source projects seeking participants) for this year's round, and Outreachy's 9 mentoring orgs also announced open internship projects.
This blog post is directed at org admins and mentors for those projects.
Many of your applicants are not fluent English writers. You have probably already experienced this, but stats back me up: Last year, GSoC had 5,199 applicants from 101 countries, many of which are not countries where English is a major medium of instruction. And nearly all the schools in the top ten were engineering schools in India, and Indian engineering schools do not teach students how to write in English at what the open source world considers a professional level. That lack of communication skills hurts your applicants as engineers, and as potential open source contributors in the long run.
I was an org admin for several years and saw, over and over, how many of our applicants had a hard time getting help and getting their ideas across because of poor writing skills. Mentors reviewed code and helped them become better coders, but weren't giving the same kind of systematic feedback about emails, bug reports, and so on, so applicants' writing skills stagnated.
In 2017, to address this, I ran English tutoring sessions for Zulip contributors. You can do this too.
Here's the call for volunteer tutors I used. Note that I explained my request in terms of global diversity and inclusion, reassured them that I'd set them up in the chatroom and be available to backchannel with them, and said "It's fine if you've never done this before and it's fine if you're not a programmer and don't know programming jargon." I circulated this request in scifi fandom, in particular in the fanfic community, which has tens of thousands of people who enjoy volunteering to proofread each other's written work and chatting online. A big source of volunteers was the Radio Free Monday weekly fandom newsletter (6 March 2017). I got 30 volunteers and was able to schedule 15 of them to tutor, and several of those volunteers were willing to do multiple 90-minute sessions.
Here's the announcement email I sent to our GSoC applicant mailing list.
We ran the tutoring sessions in the "learning" channel of our Zulip chat so it was easy to paste in links, explain proper formatting, and put side conversations in another thread. Here's the Dropbox Paper shared signup sheet where I kept the schedule and instructions for learners and tutors (basically: learners show up with a short written sample and with some thoughts about how they want to improve, and tutors take 30 minutes to critique each sample). The signup sheet format was, for example:
Date & time: Sunday, March 19th, 1:00-2:30 PM ET (10:30 PM-12 AM in India)
If only one person signed up for a session, that person got help for 45-60 minutes. Or, sometimes, we got drop-ins as other contributors got curious and realized they could ask for help on their blog posts or GSoC applications as well. After I got each tutor settled in I didn't have to pay attention for the whole 90 minutes, so I could do other Zulip work and check in occasionally -- and eventually other Zulip contributors helped out by "cohosting" so sessions could happen without me.
We ran about 20 sessions, and about 40 contributors got tutoring. They wrote better internship applications, blog posts, bug reports, code comments, pull requests, and mailing list posts because of what they learned in these sessions -- and they were so grateful for even 30 minutes of in-depth advice, because some of them had never gotten friendly, personal critique of their written English from a fluent speaker before.
So please copy me! And if several people tell me their projects are doing this, I'll help publicize your efforts together. There are a lot of fluent English writers with free time and an internet connection who would love to help the open source community in this way. Like Wikipedia, we can turn "Someone is WRONG on the Internet" into a good thing. :-)
# 25 Feb 2019, 05:52PM: API Copyrightability Law Brings (Many of) Us Together:
So you know that moment partway through a movie where heroes team up and stride toward the camera in slow motion while epic music plays?
The Python Software Foundation's counsel, Van Lindberg, writes: @ThePSF and @tidelift just filed an amicus brief urging the Supreme Court to grant cert in Oracle v. Google. This case is central to the future of free and open source software. http://www.supremecourt.gov/DocketPDF/18/18-956/89548/20190225155816527_18-956%20Amici%20Brief%20Python.pdf
Tidelift cofounder Luis Villa notes: More on this tomorrow, but very excited to join @ThePSF in this. At @tidelift we believe in growing the pie, not fighting over the scraps, and the ability to reuse and reimplement APIs is part of how the software industry has grown (and hopefully will continue to grow!)
(Amicus/amici curiae briefs always remind me of a cartoon Seth Schoen drew.)
From the brief:
The Federal Circuit's decisions are so destabilizing because they upset the settled expectations of thousands of software developers -- and particularly open source software licensees -- across all aspects of the economy.
Yup -- one reason the docket has so many other amici briefs. As far as I can tell, all of them are in favor of the Court granting certiorari.
# 03 Oct 2018, 05:16PM: Tidelift Is Paying Maintainers And, Potentially, Fixing the Economics of an Industry:
As the founder of Changeset Consulting, I keep my eye on consultancies and services in and near my niche, open source leadership, maintainership, and sustainability.* And I've known Luis Villa for years and got to work with him at Wikimedia. So yeah, I noticed when Tidelift announced its big new launch. And -- now, as a very-part-time consultant who helps Tidelift understand the Python world -- I am excited about their commitment to pay more than USD$1 million to maintainers (including "a guaranteed minimum $10,000 over the next 24 months to select projects").
Here's my take on the new Tidelift subscription model, the "lifter" role, and whom this works for.
For software businesses, this provides that missing vendor relationship, SLA, release cadence expectations, and general peace of mind for all of that unseen infrastructure you depend on. It's often easier for businesses -- of many sizes -- to pay a regular fee than to put open source project management work, dependency-updating, compliance checking, dependency security audits, or FLOSS volunteer relations on the engineering schedule.
For individual programmers and community-maintained open source projects, Tidelift is a potential source of substantial income. As a Pythonist, I hope to reach people who are currently core code contributors to open source projects in Python, especially on the Libraries.io digital infrastructure/unseen infrastructure/improve the bus factor lists. And I would like to reach projects like the ones Nathaniel Smith calls out in a recent post:
that (1) require a modest but non-trivial amount of sustained, focused attention, and (2) have an impact that is large, but broad and diffuse
and projects in the "wide open", "specialty library", and "upstream dependency" categories identified by the Open Tech Strategies report "Open Source Archetypes: A Framework For Purposeful Open Source".
For such people and projects, becoming a lifter is a promising model -- especially since the required tasks are fairly few, and are things maintainers should do anyway. I'm encouraged to see Jeff Forcier (maintainer of Fabric, Alabaster, and more) and Ned Batchelder's coverage.py getting onto the Tidelift platform.
And you can see estimated monthly income for your package right now. For some people, especially those whose healthcare doesn't depend on an employer, Tidelift payments plus some side consulting could be a sustainable, comfortable income.
Then there are folks like me whose contributions are only partially visible in commit logs (management, user support, testing, and so on), and groups that work together best as a team. Tidelift is also a potential source of income for us, but it's a little more complicated. Tidelift can send lifter payments to individuals, for-profits, and nonprofits, but: "If a package has multiple co-maintainers, you'll need to agree as a group on an approach." If you thought code of conduct conversations with your community were uncomfortable, wait till you bring up money! But, more seriously: I've been able to talk frankly with open source colleagues about thorny "who gets paid what?" questions, and if you're candid with your co-maintainers, the benefits may be pretty substantial. You can get advice on this conversation during the next live Tidelift web-based Q&A, Thursday, Oct. 11 at 2 p.m. Eastern Time (sign up at the bottom of the lifter info page).
Nonprofits, companies, and working groups that maintain projects can sign up now as lifters. Even if it's just a trickle of money right now, it might build over time and turn into enough to fund travel for an in-person sprint, contract work to improve continuous integration, an Outreachy internship, etc.
(One gap here: right now, Tidelift isn't great at supporting system-level packages and projects, like tools that get installed via apt or yum/DNF. I'm pretty sure that's something they're working on.)
What about noncommercial users or users who can't afford Tidelift subscriptions? The more lifters and subscribers sign up, the more those users benefit, too. Subscribers' funding means maintainers have time to make improvements that help everyone. And lifters agree to follow security, maintenance, and licensing best practices that also help everyone. Plus, Tidelift stewards libraries.io, a great resource for anyone who uses or develops open source (more on that). More money for Tidelift could mean libraries.io gets better too.
So I'm tooting a horn here and hoping more people sign up, because this is one of the more plausible ways open source sustainability could possibly work. Tidelift could be a real game-changer for the industry. Check it out.
* Examples: new competitors like Maintainer Mountaineer and OpenTeam, new funders like OSS Capital, and colleagues/referrals like Open Tech Strategies, VM Brasseur, Otter Tech, and Authentic Engine.
# 18 Jul 2018, 06:58PM: Libraries.io and the Infrastructure of Hospitality:
So many times in my life in open source tools and platforms, I've run into the following problem:
We want to make a breaking change/prioritize work/get feedback.
Let's check with our downstreams.
How do we find and communicate with them?
We have stopgaps inside the application (like in-app and in-API messaging, Wikipedia banners) and outside (like creating users and announce mailing lists, publicizing social media broadcast venues, searching the web/GitHub for projects and people that mention/import our tool and pinging them personally).
Now, Libraries.io makes it way easier to be hospitable to my downstreams.
I co-maintain Twine, a utility that many developers use to upload packages to PyPI. We released 1.11.0 in March. The Libraries.io page about Twine shows me who's using it (see the screenshot on the left), and I can dive deeper to check who's pinned to an old version so I can ping them to check what's up. (Sometimes that conversation tells me about a problem in our upstream dependencies that I didn't know about.) If I want to survey some users to find out whether a particular breaking change would hurt them, this makes it way easier to find some representative users to ask, not just the power users and enthusiasts who take the initiative to reach out to me.
Libraries.io has an API so I could even automate some of this. And libraries.io covers npm, CRAN, PyPI, RubyGems, Maven, and a bunch more package managers across different languages and frameworks, so I'll see downstreams that aren't just in Python.
And the code is open source in case I want to understand how they rank projects in search results or add tox.ini support in their dependency checker.
I started drafting this post in March, and in the interim, I did a little paid work for Tidelift, the company that stewards libraries.io. So, disclaimer, I'm a bit biased now. But I thought libraries.io was supercool before Tidelift and I ever thought of working together. I'm glad the Ford and Sloan Foundations funded the initial version of this tool and I'm glad Tidelift is funding and using it now; it's already very useful, and I see it as part of the infrastructure that lets maintainers and users understand each other's needs.
# 11 Apr 2018, 10:17PM: My LWN Story Summarizing PyPI's Overhaul:
This coming Monday, April 16th, we plan to flip the switch on the new PyPI and redirect https://pypi.python.org web browser requests and pip install requests so the codebase serving them is Warehouse (which is in beta right now at https://pypi.org). I'm proud of our team's work and hope you find it useful.
I haven't blogged here in a while, but I've been writing a lot, mostly announcements and explanations listed on, or a few hyperlinks away from, the onwiki index to my PyPI work. When I can't give people choices (and, unless your organization sets up a private package index/repository, PyPI can feel like the only game in town), I want to give them a lot of lead time to test, file bug reports, and migrate, and I want to provide backstory.
So: today LWN publishes a new article by me, "A new package index for Python". In it, I discuss security, policy, UX and developer experience changes in the 15+ years since PyPI's founding, new features (and deprecated old features) in Warehouse, and future plans. Plus: screenshots!
This summary should help occasional Python programmers understand why a new PyPI codebase is necessary, what's new, what features are going away, and what to expect in the near future.
If you aren't already a subscriber, you can use this subscriber link for the next week to read the article despite the LWN paywall. Thanks to LWN for the venue and the subscriber links, and thanks to Jake Edge in particular for thorough editing. Thanks to my Warehouse team for fact-checking me.
# 06 Feb 2018, 09:48AM: The Ambition Taboo As Dark Matter:
PyCon just rejected my talk submission,* so I'll try to finish and post this draft that I've been tapping at for ages.
My current half-baked theory is that programmers who want any public recognition from our peers, recognition that meaningfully validates our personal mastery, basically have to do that through one of a few fora that therefore accrue less-spoken emotional freight. And two of those places are code review in open source projects** and proposal review in tech conference talk submissions, and the fact that we don't talk enough about the role of ambition when talking about these processes leads to unnecessary hurt feelings.
For context: We give talks for varied reasons. To teach, to make reusable documentation, to show off things we've made or things we know, to burnish our credentials and thus advance our careers, to serve our corporate brands' goals, to provide role models for underindexed folks from our demographics, to give a human face to a project and make it more approachable, it goes on.
A conference talk is a tool in a toolbox that has a lot of other tools in it. (The Recompiler, Linux Journal and LWN pay for articles, for instance.)
And conferences are more than lecture halls, of course -- they're networking opportunities, communities of practice, parties, vacations, sprints, and so on.
But when we talk about the particular pain or joy of having a talk accepted or rejected from a conference, there's an emotional valence here that isn't just about the usefulness of a talk or the community of a conference. We're talking about acceptance as a species of public professional recognition.
I've found it pretty useful to think about public professional recognition in the context of Dr. Anna Fels's book Necessary Dreams. She points out that the childhood or adolescent desire for fame is often a precursor to a more nuanced ambition, combining the urge to master some domain or skill with the desire for the recognition of one's peers or community. This influences how I think about awards, about job titles, and about encouraging technologists in the public interest, and about the job market's role in skill assessment.
So how can a programmer pursue public mastery validation? Here's what I see:***
- contributing to open source software (mastery validation: maintainers merging commits and thanking/crediting contributor for work)
- presenting at conferences (mastery validation: program committee accepting talk)
- posting comments to gamified platforms like Reddit, Hacker News, and Stack Overflow (mastery validation: upvotes and replies)
- publishing academic research (mastery validation: journal accepting paper, peers reviewing paper positively)
- writing books (mastery validation: publisher accepting & publishing book)
- starting and architecting technically challenging projects (mastery validation: skilled technologists cofounding with or working for you, or relying on or praising your work)
So, this stuff is fraught; let's not pretend it's not. And we get rejected sometimes by conferences and talk about it, try to take the perspective that we're collecting "no's", we remind others that even successful and frequent speakers get rejected a lot and you can choose not to give up. And we give each other tips on how to get better at proposing talks. And that's all useful. But there's also another level of advice I want to give, to repeat something I said last year:
I try not to say "don't get discouraged," because to me that sounds like telling someone not to cry or telling someone to calm down. It's a way of saying "stop feeling what you're feeling." Instead, I try to acknowledge that something is discouraging but also -- if the other person's ready to hear it -- that we can come back from that: your feelings are legitimate, and here are some ways to work with them.
Some advice I hear about bouncing back from a conference talk rejection involves formalizing, creating systems to use to get better at writing proposals (my own tips mostly fall into this category) -- after all, in programming, you can learn to make better and better things without directly interacting with or getting feedback from individuals. The code compiles, the unit tests pass. And that can be soothing, because you can get the feedback quickly and it's likely to be a flavor of fair. (But that computer rarely initiates the celebration, never empathizes with you about the specific hard thing you're doing or have just done, and rarely autocredentials you to do something else that has a real impact on others.)
To formalize and abstract something makes it in some ways safer; it's safer to say "I'm working to pass the [test]" or "I'm building a [hard thing] implementation" or "I'm submitting a talk to [conference]" than to say "I am working to gain the professional respect of my profession". But that is one motivation for people to submit talks to tech conferences and to feel good or bad about the talks they give.
So part of my advice to you is: go ahead and be honest with yourself about how you feel. Rejection can be hard, working to get an unaccountable gatekeeper's acceptance**** and failing to get public professional recognition in your chosen field is a cause of anxiety, and so on. Be honest about how discouraging that can feel, and why, and what you wanted that you didn't get.
And another part of my advice is that I will ask, like the annoying programmer I am: what problem are you trying to solve? Because there are probably a lot of ways there that don't involve this particular gatekeeper.
And the most annoyingly empowering part of my advice is: Humans created and run PyCon and TED and Foo Camp and all the other shiny prestigious things; you're a human and you could do so too. Especially if you acknowledge not just your own but others' ambition, and leverage it.
* Maybe we'll do it in an open space anyhow.
** Another blog post for another time!
*** I've left some things out here.
We have some awards, e.g., ACM Distinguished Member, that you might get if you work really hard for decades in certain fields. That feels too far away for the kind of thing I'm thinking about.
I've left out the possibility of being promoted at your job, because many technologists perceive engineering job promotions as not particularly correlating with the quality of one's work as a programmer, which means a promotion doesn't send a strong signal, understood by peers outside one's organization, of validation of programming mastery. Then again, if your organization is old enough or big enough, maybe the career ladder there does constitute a useful proxy for the mental models of the peers whose judgment you care about.
I've left out various certifications, diplomas and badges because I don't know of any that meaningfully signal validation of one's mastery as a programmer industry-wide. And there's a lot of stuff to parse out that I feel undecided about, e.g., I find it hard to distinguish the status symbol aspect of admission from the signal that the final credential sends. And: A lot of people in this industry find it impressive when someone has been admitted to certain postsecondary engineering programs, regardless of whether the person graduates. And: In my opinion, the Recurse Center is an experience that has an unfortunate and unintended reputation for gatekeeping on the basis of programming skill, such that a big subset of people who apply and are rejected experience this as an authoritative organization telling them that they are not good enough as programmers (and Google Summer of Code and Outreachy have a related problem).
Of course, go ahead, write your own blog post where you talk about how wrong I am about what I list or exclude, especially because I come from a particular corner of the tech industry and I'm sure there's stuff I don't perceive.
**** Some conferences' gatekeepers are more unaccountable than others'; regardless, the feeling from the rejectee's point of view is, I bet, mostly the same. And you can start your own conference or join the program committee of an existing conference to see what it's like from the other side of the desk and wield a bit of the power yourself.
# 05 Jan 2018, 09:49AM: Software Freedom Conservancy Fundraiser:
I am a supporter of the Software Freedom Conservancy, a nonprofit that helps free and open source software projects. They help programmers give away their work for free. I wrote a quick neighbor-friendly introduction to what they do (video version) a few years ago, and everything in it still applies.
In case you don't want to watch/read that: Conservancy is a nonprofit umbrella (a fiscal sponsor), helping projects like Git, homebrew, Wine, Selenium, and others by taking care of legal and financial paperwork on their behalf. They're also the institutional home of Outreachy, which gives underindexed people in tech paid apprenticeship-style internships to help them start their open source software careers. And they make sure big companies actually follow the rules, legally, so everyone can benefit the same from the openness of open source.
Right now, they have a challenge match going: "All donations up to [USD] $75,000 will be matched dollar for dollar until January 15." And they're about $5,500 from reaching that goal -- we can push it over the top! Conservancy's part of the unsung infrastructure of inclusivity and fairness in open source and in the larger tech industry, and I hope those of you who can will chip in a bit.
# 02 Jan 2018, 10:46AM: 2017 Sumana In Review:
Four years ago, during my first batch at the Recurse Center, every day I'd write in a little notebook on the subway on my way home, jotting down a few bullet points about what I had learned that day. I found it helped in a variety of ways, and the keenest was that on bad days, reviewing my notes reminded me that I was in fact progressing and learning things.
On any given day in 2017 I often did not feel very happy with my progress and achievements and how I was using my time. I fell ill a lot and I was heartsick at the national political scene and current events. It is genuinely surprising to me to look back and take stock of how it all added up.
I went hiking in Staten Island and in the Hudson Valley. I got back on my bike and had some long rides, including on a canal towpath in New Jersey and over the Queensboro bridge. (And had my first accident -- a car in my neighborhood rear-ending me at a traffic light -- and thankfully escaped without damage or injury.) I learned how to bake bread. I got to meet Ellen Ullman OMG. And I tried to travel less than I had in previous years, but I still had some fine times in other places -- notably, I had a great time in Cleveland, I witnessed the total solar eclipse in Nashville, and I visited Charlotte, North Carolina (where, among other things, I visited the NASCAR Hall of Fame).
I did some of the same kinds of volunteering and activism that I'd done in previous years. For instance, I continued to co-organize MergeSort, participated in a fundraising telethon for The Recompiler telethon, signal-boosted a friend's research project to get more participants, and helped revitalize a book review community focusing on writers of color. Also, I served again as the auctioneer for the James Tiptree, Jr. Literary Award fundraising auction at WisCon, which is a particularly fun form of community service. The Tiptree Award encourages the exploration & expansion of gender. I wrote this year about what an award does, and the reflections I've seen from winners of the Tiptree Awards and Fellowships tell me those honors are doing the job -- encouraging creators and fans to expand how we imagine gender. This year I also deepened my commitment to the Tiptree Award by accepting the organization's invitation to join the Tiptree Motherboard; I am pleased to have helped the award through a donation matching campaign.
But the big change in my community service this year was that I tried to prioritize in-person political work. I called, emailed, and wrote postcards to various government officials. I participated in my local Democratic Club, including going door-to-door petitioning to get my local city councilmember onto the ballot for reelection.
And I found that I could usefully bring my technologist perspective to bear on the city and state levels, especially regarding transparency in government software. I spoke to my local councilmember about my concern regarding public access defibrillator data (the topic that led me to file my first-ever Freedom of Information Law requests, for government health department records) and this inspired him to sponsor a bill on that topic. (Which is now filed as end-of-session partly because of the limbo in potentially getting PAD data from NYC's open data portal -- I need to send an email or two.) I was invited to speak to a joint committee of the New York State Assembly on the software side of our forensics labs, and got particularly interested in this aspect of due process in our criminal justice system, publicizing the issue in my MetaFilter posts "'maybe we should throw an exception here??'" and "California v. Johnson". I testified before the Committee on Technology of the New York City Council on amendments to our open data law (I didn't prep my public comment, so this text is reconstructed from memory; video), and then spoke before the same committee on an algorithmic accountability measure (and publicized the bill, especially keeping the Recurse Center community apprised as best I could). And I did research and outreach to help ensure that a state legislature hearing on protecting the integrity of our elections included a few researchers and activists it wouldn't have otherwise.
In 2018 I want to continue on this path. I think I'm, if not making a difference, making headway towards a future where I can make a difference.
This was by far Changeset Consulting's busiest year.
I had a mix of big projects and smaller engagements. First, some of the latter: I advised PokitDok on developer engagement, with help from Heidi Waterhouse. For Open Tech Strategies, I wrote an installation audit for StreetCRM. And, working with CourageIT, I came in as a part-time project manager on a government health IT open source project so the lead developer could focus more on architecture, code, and product management.
Some larger and longer projects:
Following a sprint with OpenNews in December 2016 to help write a guide to newsrooms who want to open source their code, I worked with Frances Hocutt to create a language-agnostic, general-purpose linter tool to accompany that guide. "The Open Project Linter is an automated checklist that new (or experienced but forgetful) open source maintainers can use to make sure that they're using good practices in their documentation, code, and project resources."
I spent much of the first half of 2017 contracting with Kandra Labs to grow the Zulip community, helping plan and run the PyCon sprint and co-staffing our PyCon and OSCON booths, running English tutoring sessions alongside Google Summer of Code application prep, and mentoring an Outreachy intern, along with the usual bug triage, documentation updates, and so on. We wrapped up my work as Zulip's now such a thriving community that my help isn't as needed!
From late 2016 into 2017, I've continued to improve infrastructure and documentation for a Provider Screening Module that US states will be able to use to administer Medicaid better (the project which spurred this post about learning to get around in Java).
And just in the last few months I started working on two exciting projects with organizations close to my heart. I'm thrilled to be improving HTTPS Everywhere's project workflow for developers & maintainers over the next few months, working with Kate Chapman via Cascadia Technical Mentorship (mailing list announcement). And, thanks to funding by Mozilla's open source grants program and via the Python Software Foundation, the Python Package Index -- basic Python community infrastructure -- is getting a long-awaited overhaul. I'm the lead project manager on that effort, and Laura Hampton is assisting me. (Python milestone: my first time commenting on a PEP!)
Along the way, I've gotten a little or a lot better at a lot of things: git, bash, LaTeX, Python (including packaging), Sphinx, Read the Docs, Pandoc, regular expressions, CSS, the Java ecosystem (especially Gradle, Javadocs, Drools), the Go ecosystem, Travis CI, GitHub Pages, Postgres, sed, npm Linux system administration accessibility standards, IRC bots, and invoicing.
Talks And Other Conferences:
This year, in retrospect, instead of doing technical talks and expository lectures of the type I'm already good at, I played with form.
At LibrePlanet 2017 I gave the closing keynote address, "Lessons, Myths, and Lenses: What I Wish I'd Known in 1998" (schedule, video, in-progress transcript). I tried something aleatoric and it worked pretty well.
At Penguicon 2017 I was one of several Guests of Honor, and spoke in several sessions including "Things I Wish I'd Known About Open Source in 1998" (which was different from the LibrePlanet version, as intended) and "What If Free and Open Source Software Were More Like Fandom?" (further links).
Then, at PyGotham, Jason Owen and I co-wrote and co-starred in a play about management and code review: "Code Review, Forwards and Back" (video on YouTube, video on PyVideo, commentary).
I also attended Maintainerati and led a session, attended !!Con, worked a booth for Zulip at OSCON, attended PyCon and helped run Zulip's sprint there, and co-sponsored a post-PyGotham dinner.
Other Interesting Things I Wrote:
I did not write this year for magazines; my writing went into this blog, MetaFilter, Dreamwidth, microblogging, and client projects, mostly. I also wrote an entry for a local business competition (I didn't make it very far but I'm glad I did it, especially the finance bits) and started two book proposals I would like to return to in 2018.
I've mentioned already some of the posts I'm happy about. Some others:
"On Noticing That Your Project Is Draining Your Soul" (every once in a while someone emails me and mentions that this has helped them, which means a lot)
"How to Teach & Include Volunteers who Write Poor Patches" (12 things you can do)
"Inclusive-Or: Hospitality in Bug Tracking", a response to Jillian C. York and Lindsey Kuper.
I turned part of "Some posts from the last year on inclusion" into "Distinguishing character assassination from accountability", a post about pile-on culture and callout culture where I pulled out quotes from 11 writers on how we take/charge each other with responsibility/power within communities.
I loved Jon Bois's 17776 and discussed it with other fans on MetaFilter, and then, to try to understand its amazingness better, wrote "Boisebration", collecting links to fiction and nonfiction by Bois about class, feminism, aging, sports, politics, wonder, education, & art (and 17776 precursors/callbacks).
I found out about Robert E. Kelly, like so many did, when his kids crashed his BBC interview, then collected some links in a MetaFilter post about his writing on Korea, US foreign policy, international relations, and academia.
I wrote up a bit about "1967's most annoying question for women in Catholic ministry" on MetaFilter to signal-boost another Recurser's cool project.
I enjoyed the learning and the plot twist in "The programmer experience: redundancy edition", in which I discovered a useful resource for Form 990 filings and learned to use the Arrow library for Python date-time manipulation. And was grateful to Pro Publica.
And I made a few jokes on social media I particularly liked:
yesterday, was trying to explain virtual environments/containers/VMs to a friend and said "they range from Inception-style fake computers to putting a blanket on the floor and pretending it's lava"
today a friend and I explained leftpad & Left Shark to someone and I began sketching out a hypothetical HuffPo piece connecting them
We habitually crowdsource infrastructure from, expect unsupportedly high levels of performance from unsuspecting participants -> popcorn.gif
Public notice I received:
I got some public attention in 2017 -- even beyond the Guest of Honor and keynote speaker honors and my amazing clients -- that I would like to list, as long as I'm taking an inventory of 2017.
I rode the first revenue ride of the new Q train extension in Manhattan and really loved the art at the new 72nd Street MTA stop. A journalist interviewed me about that on video and my experience got into the New York Times story about the opening.
Presenters at the code4lib conference said their project was specifically motivated by my code4lib 2014 keynote "User Experience is a
Social Justice Issue" (written version, video). I was honored and humbled.
And -- this is out of place but I need to record it -- as someone who knew Aaron Swartz, I consented to be interviewed by artists working on a play about him, and so someone briefly portrayed me (as in, pretended to be me and repeated my words aloud) in that play, Building a Real Boy.
Finally, Hari Kondabolu looked at the English Wikipedia page about him, much of which I contributed, and was amazed at how thorough it was. So that was awesome and I was proud.
I got on Mastodon as part of my effort to improve how I use social media. I started using a new task tracker. I got back on my bike, and got somewhat into a habit of using it for some exercise and intra-city travel. A new friend got me into taking more frequent photos and noticing the world I'm in. Two new friends caused me to look for more opportunities to see musicians I love perform live.
I consumed a fair bit of media this year; didn't get into new music but enjoyed music podcasts "I Only Listen To The Mountain Goats" and "Our Debut Album". I did some book and reading reviews and will catch up to other 2017 reading sometime vaguely soon.
Leonard's film roundups & TV spotlights are a good way to see or remember most of what I saw in the last few years. TV highlights for me for 2017 are The Good Place, Jane the Virgin, The Great British Baking Show (which led me to write a tiny Asimov fanfic), Steven Universe, and Better Call Saul; I also saw Comrade Detective and Yuri!!! On Ice. Films I'm really glad I saw: The Big Sick, Schindler's List, Get Out (I fanned in MetaFilter Fanfare), In Transit, A Man For All Seasons, Hidden Figures, and Lemonade -- and a rewatch of Antitrust.
I made a few new friends this year, most notably Jason Owen and Mike Pirnat. My friends Emily and Kris got married and I got to hold up part of the chuppah for them. I took care of some friends at hard times, like accompanying them to doctor's visits. I got to see some friends I rarely see, like Mel Chua and Zed Lopez and Zack Weinberg, and kept up some old friendships by phone. My marriage is better than ever.
This year I shall iterate forward, as we all do.
# 11 Nov 2017, 08:14AM: Video of Our PyGotham Play:
You can now watch the 22-minute video of the play I discussed last month. "Code Review, Forwards and Back", co-written by and co-starring Jason Owen and me, directed by Jonathan Galvez.
- Kenneth Durril for running sound
- David Beazley for running lights (on a few hours' notice and with no rehearsal)
- A. Jesse Jiryu Davis for a cameo as a junior engineer, and for introducing the play
- Jonathan Galvez for directing (if you're in NYC and looking to hire a director for a thing like this, ask me for his email address)
- Michael Rehse for a ton of useful advice
- Laura Hampton for serving as a dramaturg during late rehearsals
- The PyGotham organizers for accepting the talk and advising us on logistics and tone
- Our audience, especially attendees who told us they'd liked it
We were happy to hear people say things like I'm new to the industry, and this helped me learn things to watch out for or I used to be that reviewer and I'm trying not to be anymore or My name is Randall and I never hear my name in fiction and it was nice to hear you say my name or I don't code at all but this is a marvelous management parable. Indeed, code review is just a particularly visible moment where you can see the effects of an organization's culture and processes. Too execution-focused (the right hand doesn't know what the left hand is doing)? Too alignment-focused (we're taking so much time deliberating and gaining consensus that we can't make forward progress on the mission)? Too lax, or too superficial, in enforcing rules? Our play can't dive into every scenario but it's a start. And -- the most frequent comment we got from happy attendees -- it was a change of pace (no slides!).
We're revising the play and submitting this a few other places; once it's run its course, we'll be posting the text of the script online.
# (2) 13 Jun 2017, 10:41AM: Transparency And Accountability In Government Forensic Science:
In February, I learned that the New York State Assembly was planning a public hearing on government oversight of forensic science laboratories, and then was invited to offer ten minutes of testimony and then answer legislators' questions. This was a hearing held jointly by the Assembly Standing Committees on Codes, on Judiciary, and on Oversight, Analysis and Investigation and it was my first time speaking in this sort of capacity. I spoke on the importance of auditability and transparency in software used in devices the government uses in laboratories and field tests, and open source as an approach to improve these. And I testified to the efficiency, cost savings, security, and quality gains available by using open source software and by reusing and sharing open source software with other state governments. Here's a PDF of my testimony as written, and video and audio recordings are available as is a transcript that includes answers to the legislators' questions. It is a thrilling feeling to see my own words in a government hearing transcript, in that typeface and with those line numbers!
As I was researching my testimony, I got a lot of help from friends who introduced me to people who work in forensics or in this corner of the law. And I found an article by lawyer Rebecca Wexler on the danger of closed-source, unauditable code used in forensic science in the criminal justice system, and got the committee to also invite her to testify. Her testimony's also available in the recordings and transcript I link to above. And today she has a New York Times piece, "How Computers Are Harming Criminal Justice", which includes specific prescriptions:
Defense advocacy is a keystone of due process, not a business competition. And defense attorneys are officers of the court, not would-be thieves. In civil cases, trade secrets are often disclosed to opposing parties subject to a protective order. The same solution should work for those defending life or liberty.
The Supreme Court is currently considering hearing a case, Wisconsin v. Loomis, that raises similar issues. If it hears the case, the court will have the opportunity to rule on whether it violates due process to sentence someone based on a risk-assessment instrument whose workings are protected as a trade secret. If the court declines the case or rules that this is constitutional, legislatures should step in and pass laws limiting trade-secret safeguards in criminal proceedings to a protective order and nothing more.
I'll add here something I said during the questions-and-answers with the legislators:
And talking about the need for source code review here, I'm going to speak here as a programmer and a manager. Every piece of software that's ever been written that's longer than just a couple of lines long, that actually does anything substantive, has bugs. It has defects. And if you want to write code that doesn't have defects or if you want to at least have an understanding of what the defects are so that you can manage them, so that you can oversight them (the same way that we have a system of democracy, right, of course there's going to be problems, but we have mechanisms of oversight) -- If in a system that's going to have defects, if we don't have any oversight, if we have no transparency into what those instructions are doing and to what the recipe is, not only are we guaranteed to have bugs; we're guaranteed to have bugs that are harder to track down. And given what we've heard earlier about the fact that it's very likely that in some of these cases there will be discriminatory impacts, I think it's even more important; this isn't just going to be random.
I'll give you an example. HP, the computer manufacturer, they made a web camera, a camera built into a computer or a laptop that was supposed to automatically detect when there was a face. It didn't see black people's faces because they hadn't been tested on people with darker skin tones. Now at least that was somewhat easy to detect once it actually got out into the marketplace and HP had to absorb some laughter. But nobody's life was at stake, right?
When you're doing forensic work, of course in a state the size of New York State, edge cases, things that'll only happen under this combination of combination of conditions are going to happen every Tuesday, aren't they? And the way that the new generation of probabilistic DNA genotyping and other more complex bits of software work, it's not just: Okay, now much of fluid X is in sample Y? It's running a zillion different simulations based on different ideas of how the world could be. Maybe you've heard like the butterfly effect? If one little thing is off, you know, we might get a hurricane.
# (1) 07 Apr 2017, 03:36PM: Inclusive-Or: Hospitality in Bug Tracking:
Lindsey Kuper asked:
I’m interested in hearing about [open source software] projects that have successfully adopted an "only insiders use the issue tracker" approach. For instance, a project might have a mailing list where users discuss bugs in an unstructured way, and project insiders distill those discussions into bug reports to be entered into the issue tracker. Where does this approach succeed, and where does it fail? How can projects that operate this way effectively communicate their expectations to non-insider users, especially those users who might be more accustomed to using issue trackers directly?
More recently, Jillian C. York wrote:
...sick of "just file a bug with us through github!" You realize that's offputting to your average users, right?
If you want actual, average users to submit bugs, you know what you have to do: You have to use email. Sorry, but it's true.
Oh, and that goes especially for high-risk users. Give them easy ways to talk to you. You know who you are, devs.
Both Kuper and York get at: How do we open source maintainers get the bug reports we need, in a way that works for us and for our users?
My short answer is that open source projects should have centralized bug trackers that are as easy as possible to work in as an expert user, and that they should find automated ways to accept bug reports from less structured and less expert sources. I'll discuss some examples and then some general principles.
Dreamwidth: Dreamwidth takes support questions via a customer support interface. The volunteers and paid staff answering those questions sometimes find that a support request reveals a bug, and then file it in GitHub on the customer's behalf, then tell her when it's fixed. (Each support request has a private section that only Support can see, which makes it easier to track the connection between Support requests and GitHub issues, and Support regulars tend to have enough ambient awareness of both Support and GitHub traffic to speak up when relevant issues crop up or get closed.) Dreamwidth users and developers who are comfortable using the GitHub issue tracker are welcomed if they want to file bugs there directly instead.
Dreamwidth also has a non-GitHub interface for feature suggestions: the suggestions form is the preferred interface for people to suggest new features for Dreamwidth. Users post their suggestions into a queue and a maintainer chooses whether to turn that suggestion into a post for open discussion in the dw-suggestions community, or whether to bounce it straight into GitHub (e.g., for an uncontroversial request to whitelist a new site for media embedding or add a new site for easy cross-site user linking, or at the maintainer's prerogative). Once a maintainer has turned a suggestion into a post, other users use an interface familiar to them (Dreamwidth itself) to discuss whether they want the feature. Then, if they and the maintainer come to consensus and approve it, the maintainer adds a ticket for it to GitHub. That moderation step has been a bottleneck in the past, and the process of moving a suggestion into GitHub also hasn't yet been automated.
Since discussion about site changes needs to include users who aren't developers, Dreamwidth maintainers prefer that people use the suggestions form; experienced developers sometimes start conversations in GitHub, but the norm (at least the official norm) is to use dw-suggestions; I think the occasional GitHub comment suffices for redirecting these discussions.
Zulip: We use GitHub issues. The Zulip installations hosted by Kandra Labs (the for-profit company that stewards the open source project) also have a "Send feedback" button in one of the upper corners of the Zulip web user interface. Clicking this opens a private message conversation with feedback-at-zulip.com, which users used more heavily when the product was younger. (We also used to have a nice setup where we could actually send you replies in-Zulip, and may bring that back in the future.)
I often see Tim Abbott and other maintainers noticing problems that new users/customers are having and, while helping them (via the zulip-devel mailing list, via the Zuliping-about-Zulip chat at chat.zulip.org, or in person), opening GitHub issues about the issue, as the next step towards a long-term fix. But -- as with the Dreamwidth example -- it is also fine for people who are used to filing bug reports or feature requests directly to go ahead and file them in GitHub. And if Tim et alia know that the person they're helping has that skill and probably has the time to write up a quick issue, then the maintainers will likely say, "hey would you mind filing that in GitHub?"
We sometimes hold live office hours at chat.zulip.org. At yesterday's office hour, Tim set up a discussion topic named "warts" and said,
I think another good topic is to just have folks list the things that feel like they're some of our uglier/messier parts of the UI that should be getting attention. We can use this topic to collect them :).
Several people spoke up about little irritations, and we ended up filing and fixing multiple issues. One of Zulip's lead developers, Steve Howell, reflected: "As many bug reports as we get normally, asking for 'warts' seems to empower customers to report stuff that might not be considered bugs, or just empower them to speak up more." I'd also point out that some people feel more comfortable responding to an invitation in a synchronous conversation than initiating an asynchronous one -- plus, there's the power of personal invitation to consider.
As user uptake goes up, I hope we'll also have more of a presence on Twitter, IRC, and Stack Overflow in order to engage people who are asking questions there and help them there, and get proto-bug reports from those platforms to transform into GitHub issues. We already use our Twitter integration to help -- if someone mentions Zulip in a public Tweet, a bot tells us about it in our developers' livechat, so we can log into our Twitter account and reply to them.
MediaWiki and Wikimedia: Wikipedia editors and other contributors have a lot of places they communicate about the sites themselves, such as the technical-issues subforum of English Wikipedia's "Village Pump", and similar community-conversation pages within other Wikipedias, Wikivoyages, etc. Under my leadership, the team within Wikimedia Foundation's engineering department that liaised with the larger Wikimedia community grew more systematic about working with those Wikimedia spaces where users were saying things that were proto-bug reports. We got more systematic about listening for those complaints, filing them as bugs in the public bug tracker, and keeping in touch with those reporters as bugs progressed -- and building a kind of ambassador community to further that kind of information dissemination. (I don't know how well that worked out; I think we built a better social infrastructure for people who were already doing that kind of volunteer work ad hoc, but I don't know whether we succeeded in recruiting more people to do it, and I haven't kept a close eye on how that's gone in the years since I left.)
We also worked to make it easy for people to report bugs into the main bug tracker. The Bugzilla installation we had for most of the time that I was at Wikimedia had two bug reporting forms: a "simple" submission form that we pointed most people to, with far fewer fields, and an "advanced" form that Wikimedia-experienced developers used. They've moved to Phabricator now, and I don't know whether they've replicated that kind of two-lane approach.
A closed-source example: FogBugz. When I was at Fog Creek Software doing sales and customer support, we used FogBugz as our internal bug tracker (to manage TODOs for our products,* and as our customer relationship manager). Emails into the relevant email addresses landed in FogBugz, so it was easy for me to reply directly to help requests that I could fix myself, and easy for me to note "this customer support request demonstrates a bug we need to fix" and turn it into a bug report, or open a related issue for that bug report. If I recall correctly, I could even set the visibility of the issue so the customer could see it and its progress (unusual, since almost all our issue-tracking was private and visible only within the company).
An interface example: Debian. Debian lets you report bugs via email and via the command-line reportbug program. As the "how to use BTS" guide says,
some spam messages managed to send mails to -done addresses. Those are usually easily caught, and given that everything can get reverted easily it's not that troublesome. The package maintainers usually notice those and react to them, as do the BTS admins regularly.
The BTS admins also have the possibility to block some senders from working on the bug tracking system in case they deliberately do malicious things.
But being open and inviting everyone to work on bugs totally outweighs the troubles that sometimes pop up because of misuse of the control bot.
And that leads us to:
General guidelines: Dreamwidth, Zulip, MediaWiki, and Debian don't discourage people from filing bug reports in the official central bug tracker. Even someone quite new to a particular codebase/project can file a very helpful and clear bug report, after all, as long as they know the general skill of filing a good bug report. Rather, I think the philosophy is what you might find in hospitable activism in general: meet people where they are, and provide a means for them to conveniently start the conversation in a time, place, and manner that's more comfortable for them. For a lot of people, that means email, or the product itself.
Failure modes can include:
- a disconnect among the different "places" such that the central bug tracker is a black hole and nothing gets reported back to the more accessible place or the original reporter
- a feeling of elitism where only special important people are allowed to even comment in the main bug tracker
- bottlenecks such that it seems like there's a non-bug-tracker way to report a question or suggestion but that process has creaked to a halt and is silently blocking momentum
- bottlenecks in bug triage
- brusque reaction at the stage where the bug report gets to the central bug tracker (e.g., "oh that's a duplicate; CLOSE" without explanation or thanks), which jars the user (who's expecting more explicit friendliness) and which the user perceives as hostile
Whether or not you choose to increase the number of interfaces you enable for bug reporting, it's worth improving the user experience for people reporting bugs into your main bug tracker. Tedious, lots-of-fields issue tracker templates and UIs decrease throughput, even for skilled bug reporters who simply aren't used to the particular codebase/project they're currently trying to file an issue about. So we should make that easier. You can provide an easy web form, as Wikimedia did via the simplified Bugzilla form, or an email or in-application route, as Debian does.
And FLOSS projects oughta do what the Accumulo folks did for Kuper, too, saying, "I can file that bug for you." We can be inclusive-or rather than exclusive-or about it, you know? That's how I figure it.
* Those products were CityDesk, Copilot, and FogBugz -- this was before Kiln, Stack Overflow, Trello, and Glitch.
Thanks to Lindsey Kuper and Jillian C. York for sparking this post, and thanks to azurelunatic for making sure I got Dreamwidth details right.
# (1) 04 Apr 2017, 12:37PM: How to Teach And Include Volunteers who Write Poor Patches:
You help run an open source software community, and you've successfully signalled that you're open to new contributors, including people who aren't professional software engineers. And you've already got an easy developer setup process and great test coverage so it's easy for new people to get up and running fast. Great!
Some of the volunteers who join you are less-skilled programmers, and they're submitting pull requests/patches that need a lot of review and reworking before you can merge them.
How do you improve these volunteers' work, help them do productive things for the project, and encourage and include them?
My suggestions for you fall into three categories: helping them
improve their code, dealing with the poor-quality pull requests
themselves, and redirecting their energies to improve the project in other ways.
Teaching them to improve their code
- Collect and suggest relevant learning resources, like certain talk recordings or freely available articles/exercises (e.g. The Architecture of Open Source Applications), and ask them to come back after they've watched/read/done them. Example: Zulip's collection.
- If developers have trouble writing good comments and commit messages, or diving into the codebase to find relevant files and commits, point them to my blog post "On the scientific method and usable history". It explains why it's important to do that, and gives them pointers.
- Ask more experienced contributors to pair program with them, both as leader and as follower. Here are a few tools to help.
- Run live coding exercises, over chat or video, where an experienced developer speaks aloud as she writes a bugfix, including all the little steps like searching for related commits, setting up and running tests, etc. This enables newer developers to learn a lot of tips that help them work faster and write higher-quality code. I've done this at Wikimedia with live video and we use Zulip for a live text approach (see Alicja Raszkowska's transcript and notes of one such session).
- If a big problem with their submissions is poor English writing skills, run some English tutoring sessions.
Dealing with poor patches themselves
Using their knowledge and curiosity to improve the project in other ways
This list is absolutely not the be-all and end-all for this topic; I'd like to know what approaches others use.
- Ask these developers to write "discovery reports". They're already user-testing your developer onboarding process; ask them for their experiences, so you can find and fix pain points.
- Ask them to run through some manual testing (example manual testing guide from Zulip), and to tell you how long certain kinds of tests took, so you can get bug reports and improve the docs.
- Ask them to teach about your project in their communities -- to develop learning and presentation materials and speak at meetups. You may have just found your most enthusiastic marketer.
Thanks to Noah Swartz for starting a conversation at Maintainerati that spurred me to write this post.
# 07 Feb 2017, 11:18AM: Upcoming Talks:
I happened upon the New York state Assembly's website last week,* and noticed an upcoming hearing about "Government oversight of forensic science laboratories" (PDF), hearing oral testimony by invitation only. I wondered: Who's on the list of witnesses? And will any of them talk about the danger of closed-source, unauditable code used in forensic science in the criminal justice system?
I followed up, and we got me, plus Rebecca Wexler, the author of that piece, invited to speak. We're testifying tomorrow, Wednesday, February 8th, in New York City. In preparation, I'm conferring with Karen Sandler of Software Freedom Conservancy (who was slotted to speak but now can't) and with acquaintances who work in government forensic labs.
I did speech and debate in high school so in some sense I have been preparing for this for twenty years.
A little further off:
Next week, I will participate in the WONTFIX Cabal (Maintainerati) unconference for open source maintainers on February 15, 2017, in San Francisco, California, USA.
I will give the closing keynote address at LibrePlanet, a free software conference, March 25-26, 2017, in Cambridge, Massachusetts, USA. Tentative title: "Lessons, Myths, and Lenses: What I Wish I'd Known in 1998."
I will be one of the Guests of Honor at Penguicon, an open source and science fiction convention, April 28-30 2017, in Southfield, Michigan, USA.
* via Lauren Sperber's blog post about "the New York State Reproductive Health Act to get abortion removed from New York State's Penal Code"
# 27 Dec 2016, 11:43AM: Yuletide 2016 Recommendations:
Every year the Yuletide fanfic exchange delivers a bounty of fun transformative works concerning books, movies, songs, games, news stories, and other parts of our media landscape. I myself have, as they say, committed fanfic a few times, but right now I'm much more a reader and cheerleader than a fiction-writer. I have only started on this year's harvest but I already have some favorites to recommend:
A hopeful story, using "Expert judgment on markers to deter inadvertent human intrusion into the Waste Isolation Pilot Plant" (you know, "Sending this message was important to us. We considered ourselves to be a powerful culture.") to tell a ghost map story. (If you want more hope about far future human civilizations, try the fanvid "Dance Apocalyptic" which cheered me this year. And here's more fic about those waste markers.)
This fairy tale, about children and destiny, stands alone so you can read it even if you've never looked at the illustrations that inspire it.
There was once a land, long before and far away from these troubled times, where every child was born with a desire and a destination marked upon them, so that they might know what dwelt in their future. Upon their left hand, a symbol to represent what would give them the greatest happiness in their life. And upon their right hand, a compass that would lead them in the direction of where their desire might be found.
If you liked Hail, Caesar!, perhaps you wanted to revel in the loveliness of Hobie Doyle, who is an understated instance of the Captain Carrot/Middleman/Captain America/Agent Dale Cooper archetype.
The Ghostbusters get a call to a theater built in 1925, and Patty Tolan really shines.
"The War of the Worlds and All That" is a Jeeves and Wooster story that has aliens and mentions Gussie Fink-Nottle and the scripture knowledge prize Bertie won in school, and it's a bunch of fun. And if you're missing the sartorial scheming, enjoy "Jeeves and the Christmas Socks". (I grew up on Wodehouse and on the Fry and Laurie adaptations -- relatedly, here's a sweet story about Tony and Control.)
It's been a while since I read Jurassic Park but "A Strange Attractor in a Stable System" gets Ian Malcolm's voice so right.
If you enjoyed the 1941 movie Ball of Fire (particularly relevant to Wikipedians, incidentally), how about a crossover story that includes The Middleman? And, speaking of The Middleman, "The Extraterrestrial Elf Emergency" includes a paragraph I adore:
"We don't have Christmas on my planet," they said plaintively, through a translator box at the base of their throat. "All our holidays are about military victories and death. Christmas seemed fun."
This Mulan story makes the Disney movie make more sense in ways I had not even thought before.
If you enjoyed Good Omens then perhaps you will like one or more of the three different stories in which those characters enact their own version of "The Devil Went Down to Georgia".
I've never seen the 1944 film Gaslight but this story, set after the film, is about bravery and recovery and resilience and I drank it deep and felt nourished.
No, she thought. I must stop being afraid and bear this until it is done and then, then I'll consider what to do next.
I also enjoyed stories transforming Mrs. Piggle-Wiggle, Fresh Off The Boat, Arrested Development, Arrival, Baby-Sitters Club, and the Mahabharata. And I haven't finished this year's Yuletide yet. Thank you, authors and organizers!
# (1) 12 Oct 2016, 11:00AM: Rough Notes for New FLOSS Contributors On The Scientific Method and Usable History:
Some thrown-together thoughts towards a more comprehensive writeup. It's advice on about how to get along better as a new open source participant, based on the fundamental wisdom that you weren't the first person here and you won't be the last.
We aren't just making code. We are working in a shared workplace, even if it's an online place rather than a physical office or laboratory, making stuff together. The work includes not just writing functions and classes, but experiments and planning and coming up with "we ought to do this" ideas. And we try to make it so that anyone coming into our shared workplace -- or anyone who's working on a different part of the project than they're already used to -- can take a look at what we've already said and done, and reuse the work that's been done already.
We aren't just making code. We're making history. And we're making a usable history, one that you can use, and one that the contributor next year can use.
So if you're contributing now, you have to learn to learn from history. We put a certain kind of work in our code repositories, both code and notes about the code. git grep idea searches a code repository's code and comments for the word "idea", git log --grep="idea" searches the commit history for times we've used the word "idea" in a commit message, and git blame codefile.py shows you who last changed every line of that codefile, and when. And we put a certain kind of work into our conversations, in our mailing lists and our bug/issue trackers. We say "I tried this and it didn't work" or "here's how someone else should implement this" or "I am currently working on this". You will, with practice, get better at finding and looking at these clues, at finding the bits of code and conversation that are relevant to your question.
And you have to learn to contribute to history. This is why we want you to ask your questions in public -- so that when we answer them, someone today or next week or next year can also learn from the answer. This is why we want you to write emails to our mailing lists where you explain what you're doing. This is why we ask you to use proper English when you write code comments, and why we have rules for the formatting and phrasing of commit messages, so it's easier for someone in the future to grep and skim and understand. This is why a good question or a good answer has enough context that other people, a year from now, can see whether it's relevant to them.
Relatedly: the scientific method is for teaching as well as for troubleshooting. I compared an open source project to a lab before. In the code work we do, we often use the scientific method. In order for someone else to help you, they have to create, test, and prove or disprove theories -- about what you already know, about what your code is doing, about the configuration on your computer. And when you see me asking a million questions, asking you to try something out, asking what you have already tried, and so on, that's what I'm doing. I'm generally using the scientific method. I'm coming up with a question and a hypothesis and I'm testing it, or asking you to test it, so we can look at that data together and draw conclusions and use them to find new interesting questions to pursue.
So I'll ask a question to try and prove or disprove my hypothesis. And if you never reply to my question, or you say "oh I fixed it" but don't say how, or if you say "no that's not the problem" but you don't share the evidence that led you to that conclusion, it's harder for me to help you. And similarly, if I'm trying to figure out what you already know so that I can help you solve a problem, I'm going to ask a lot of diagnostic questions about whether you know how to do this or that. And it's ok not to know things! I want to teach you. And then you'll teach someone else.
- Expected result: doing run-dev.py on your machine will give you the same results as on mine.
- Actual observation: you get a different result, specifically, an error that includes a permissions problem.
- Hypothesis: the relevant directories or users aren't set up with the permissions they need.
- Next step: Request for further data to prove or disprove hypothesis.
In our coding work, it's a shared responsibility to generate hypotheses and to investigate them, to put them to the test, and to share data publicly to help others with their investigations. And it's more fruitful to pursue hypotheses, to ask "I tried ___ and it's not working; could the reason be this?", than it is to merely ask "what's going on?" and push the responsibility of hypothesizing and investigation onto others.
This is a part of balancing self-sufficiency and interdependence. You must try, and then you must ask. Use the scientific method and come up with some hypotheses, then ask for help -- and ask for help in a way that helps contribute to our shared history, and is more likely to help ensure a return-on-investment for other people's time.
So it's likely to go like this:
- you try to solve your problem until you get stuck, including looking through our code and our documentation, then start formulating your request for help
- you ask your question
- someone directs you to a document
- you go read that document, and try to use it to answer your question
- you find you are confused about a new thing
- you ask another question
- now that you have demonstrated that you have the ability to read, think, and learn new things, someone has a longer talk with you to answer your new specific question
- you and the other person collaborate to improve the document that you read in step 4 :-)
This helps us make a balance between person-to-person discussion and documentation that everyone can read, so we save time answering common questions but also get everyone the personal help they need. This will help you understand the rhythm of help we provide in livechat -- including why we prefer to give you help in public mailing lists and channels, instead of in one-on-one private messages or email. We prefer to hear from you and respond to you in public places so more people have a chance to answer the question, and to see and benefit from the answer.
We want you to learn and grow. And your success is going to include a day when you see how we should be doing things better, not just with a new feature or a bugfix in the code, but in our processes, in how we're organizing and running the lab. I also deeply want for you to take the lessons you learn -- about how a group can organize itself to empower everyone, about seeing and hacking systems, about what scaffolding makes people more capable -- to the rest of your life, so you can be freer, stronger, a better leader, a disruptive influence in the oppressive and needless hierarchies you encounter. That's success too. You are part of our history and we are part of yours, even if you part ways with us, even if the project goes defunct.
This is where I should say something about not just making a diff but a difference, or something about the changelog of your life, but I am already super late to go on my morning jog and this was meant to be a quick-and-rough braindump anyway...
# 04 Aug 2016, 03:51PM: Advice on Starting And Running A New Open Source Project:
Recently, a couple of programmers asked me for advice on starting and running a new open source project. So, here are some thoughts, assuming you're already a programmer, you haven't led a team before, and you know your new software project is going to be open source.
I figure there are a few different kinds of best practices in starting and running open source projects.
General management: Some of my recommendations are the same kinds of best practices that are useful anytime you're starting/running/managing any kind of project, inside or outside the software world.
For instance: know why you're starting this thing. Write down even just a one-paragraph or 100-word bulleted list description of what you are aiming at. This will reduce the chance that you'll look up one day and see that your targeted little tool has turned into a mess that's trying to be an entire operating system.
And: if you're making something that you want other people to use, then check what those other people are already using/doing, so you can make sure you suit their needs. This guards against any potential perception that you are starting a new project thoughtlessly, or just for the heck of it, or to learn a new framework. In the software world, this includes taking note of your target users' dependencies (e.g., the versions of Python/NumPy that they already have installed).
Resources I have found useful here include William Ball's book on theatrical direction A Sense of Direction, Dale Carnegie's How to Win Friends and Influence People, Fisher & Ury's Getting To Yes, Cialdini's Influence: The Science of Persuasion, and Ries & Trout's Positioning: The Battle for Your Mind.
Tech management: Some best practices are the same kinds of habits that help in managing any kind of software project, including closed-source projects as well.
For instance: more automated tests in/for your codebase are better, because they reduce regressions so you can move faster and merge others' code faster (and let others review and merge code faster), but don't sweat getting to 100%, because there's definitely a decreasing marginal utility to this stuff. Travis CI is pretty easy to set up for the common case.
I assume you're using Git. Especially if you're going to be the maintainer on a code level, learn to use Git beyond just push and pull. Clone a repo of a project you don't care about and try the more advanced commands as you make little changes to the code, so if you ruin everything you haven't actually set your own work back. Learn to branch and merge and work with remotes and cherry-pick and bisect. Read this super useful explanation of the Git model which articulates what's actually doing what -- it helps.
Good resources here include Brooks's The Mythical Man-Month, DeMarco & Lister's Peopleware, Heidi Waterhouse's "The Seven Righteous Fights", Camille Fournier's blog, and my own talk "Learn Tech Management in 45 Minutes" and my article "Software in Person". I myself earned a master's in technology management and if you are super serious about becoming a technology executive then that's a path I can give more specific thoughts on, but I'm not about to recommend that amount of coursework to someone who isn't looking to make a career out of this.
Open source management: And some best practices are the specific social, product management, architectural, and infrastructural best practices of open source projects. A few examples:
If you're the maintainer, it's key to reply to new project-related emails, queries, bug reports, and patches fast; a Mozilla analysis backs up our experience that a kind, fast, negative response is better than a long silent delay. Reply to people fast, even if it's just "I saw this, thank you, I'm busy, will get to this in a few weeks," because otherwise the uncertainty is deathly and people's enthusiasm and momentum drip away.
Make announcements somewhere public and easily findable that say something about the current state of your project, e.g., about whether it's ready to use or when to expect it to be. This could even just be someplace prominent in your README when you're just getting started. This is also a good place to mention if you're going to be at any upcoming conferences, so people can connect to you that way.
Especially when it comes to code, docs, and bug/feature/task lists, work in the open from as early as possible, preferably from the start. Treat private work as a special case (sometimes a useful one when it comes to communication with users and with new contributors, as a tidepool incubates growth that can then flow into the ocean).
I am sad, as a FLOSS zealot, to say that you should probably be on the closed-source platform that is GitHub. But yeah, the intake funnel for code and bug contributors is easier on GitHub than on any other platform; unless you are pretty sure you already know who all the people are who will use and improve this software, and they're all happy on GitLab or similar, GitHub is going to get you more and faster contributors.
You are adjacent to or embedded in other programming communities, like the programming language & frameworks you're using. Use the OSI-approved license that the projects you're adjacent to/depending on use, to make reuse easier.
It's never too early to think about governance. As Christie Koehler of Authentic Engine warns, to think about codes of conduct, you also gotta think about governance. (The Contributor Covenant is a popular starting point.) If you can be under the umbrella of a software-related nonprofit, like NumFOCUS, that'll help you make and implement these choices.
Top reading recommendation: Karl Fogel's Producing OSS is basically the bible for this category, and the online version is up-to-date with new advice from this year. If you read Producing OSS cover-to-cover you will be entirely set to start and run your project.
Additionally: Fogel also co-wrote criteria for assessing whether a project "is created and managed in a sustainably open source way". And I recommend my own blog post "How To Improve Bus Factor In Your Open Source Project", the Linux Foundation CII criteria (hat-tip to Benjamin Gilbert), "build your own rockstars" by one of the founders of the Dreamwidth project, and "dreamwidth as vindication of a few cherished theories" by that same founder (especially the section starting "our development environment and how we managed to create a process and culture that's so welcoming").
Obligatory plug: I started Changeset Consulting, which provides targeted project management and release management services for open source projects and the orgs that depend on them. In many ways I am maintainer-as-a-service. If you want to talk more about this work, please reach out!
# 21 Mar 2016, 05:20PM: Two Conferences, Three Talks:
Last week I took the train to Atlanta to speak at the Great Wide Open conference, which I'd never visited before. I particularly appreciated the chance to share my lessons learned with an audience that was diverse on gender, ethnic, speciality, and other dimensions, the cozy and delightful speakers' dinner, and the organizing team's consistent approachability and helpfulness. If Atlanta is an easy trip for you, and if you're interested in growing your skills in free and open source software, I suggest you consider attending next year.
I spoke on underappreciated features in HTTP, and my slides are available as a PDF. If you're going to be at PyCon North America in Portland, Oregon this year, I'll be presenting a more Python-specific version of this talk there on May 31st. If neither of those works for you, check out the video of the very similar "HTTP Can Do That?!" presentation I delivered at Open Source Bridge last year.
Then I rode the train north to Boston; along the way I got to converse with a neat seatmate, a military veteran who loves taking family walks after dinner to play Ingress with his kids. Awww. His son loves Minecraft so I got to recommend the NYPL historical-map Minecraft worlds to him.
Then, this past weekend, I attended my first LibrePlanet. What a lovely time I had! I saw rockin' talks by people whose thoughts I was already eager to hear, and I met dozens of people who are working on promising projects like a nonprofit, transparent search for the web and a browser extension that lets users share their internet connections with people whose connections are censored. I especially commend the organizers for running the conference, including the video streaming, using entirely free and open source software! Since we knew that all of us are dedicated to software freedom as a goal in itself and towards a more just and a freer world, we could have complex conversations that advanced beyond first-contact advocacy and into details and long-term planning.
I spoke on "Inessential Weirdnesses in Free Software"; the written remarks I spoke from are now available as a text file. It is not the most legible page in the world, because I will be further revising this talk before presenting it at OSCON in Austin, Texas, on May 18th.
I also delivered a somewhat impromptu five-minute lightning talk, "What is maintainership? Or, approaches to filling management skill gaps in free software". I've now posted the textual version of that talk.
LibrePlanet participants told me they really liked both my talks; the latter especially spurred some to talk with me about potential contracts with my firm, Changeset Consulting, which was a big morale injection since I'm definitely seeking leads and referrals right now.
Thanks to both LibrePlanet and Great Wide Open for having me speak! I've also updated my Talks webpage with links to my upcoming appearances. My calendar's open after August, in case you know anyone looking for a speaker who makes a lot of jokes and gestures.
# 21 Mar 2016, 04:58PM: What Is Maintainership?:
Yesterday, at my first LibrePlanet conference, I delivered a somewhat impromptu five-minute lightning talk, "What is maintainership? Or, approaches to filling management skill gaps in free software". I spoke without a script, and what follows is what I meant to say; I hope it bears a strong resemblance to what I actually said. I do not know whether any video of this session will appear online; if it does, I'll update this entry.
What is Maintainership?
Or, approaches to filling management skill gaps in free software
Sumana Harihareswara, Changeset Consulting
LibrePlanet, Cambridge, MA, 20 March 2016
Why do we have maintainers in free software projects? There are various different explanations you can use, and they affect how you do the job of maintainer, how you treat maintainers, how and whether you recruit and mentor them, and so on.
So here are three -- they aren't the only ways people think about maintainership, but these are three I have noticed, and I have given them alliterative names to make it easier to think about and remember them.
Sad: This is a narrative where even having maintainers is, fundamentally, an admission of failure. Jefferson said a lot of BS, but one thing he said that wasn't was: "If men were angels, we would have no need of government." And if every contributor contributed equally to bug triage, release management, communication, and so on, then we wouldn't need to delegate that responsibility to someone, to a maintainer. But it's not like that, so we do. It's an approach to preventing the Tragedy of the Commons.
I am not saying that this approach is wrong. It's totally legitimate if this is how you are thinking about maintainership. But it's going to affect how your community does it, so, just be aware.
Skill: This approach says, well, people want to grow their skills. This is really natural. People want to get better, they want to achieve mastery, and they want validation of their mastery, they want other people to respect their mastery. And the skill of being a maintainer, it's a skill, or a set of skills, around release management, communication, writing, leadership, and so on. And if it's a skill, then you can learn it. We can mentor new maintainers, teach them the skills they need.
So in this approach, people might have ambition to be maintainers. And ambition is not a dirty word. As Dr. Anna Fels puts it in her book Necessary Dreams, ambition is the combination of the urge to achieve mastery of some domain and the desire to have your peers, or people you admire, acknowledge, recognize, validate your mastery.
With this skills approach, we say, yeah, it's natural that some people have ambition to get better as developers and also to get better at the skills involved in being a maintainer, and we create pathways for that.
Sustain: OK, now we're talking about the economics of free software, how it gets sustained. If we're talking about economics, then we're talking about suppply and demand. And I believe that, in free software right now, there is an oversupply of developers who want to write feature code, relative to an undersupply of people with the temperament and skills and desire to do everything else that needs doing, to get free software polished and usable and delivered and making a difference. This is because of a lot of factors, who we've kept out and who got drawn into the community over the years, but anyway, it means we don't have enough people who currently have the skill and interest and time to do tasks that maintainers do.
But we have all these companies, right? Companies that depend on, that are built on free software infrastructure. How can those with more money than time help solve this problem?
[insert Changeset Consulting plug here]. You can hire my firm, Changeset Consulting, to do these tasks for a free software project you care about. Changeset Consulting can do bug triage, doc rewriting, user experience research, contributor outreach, release management, customer service, and basically all the tasks involved in maintainership except for the writing and reviewing of feature code, which is what those core developers want to be doing anyway. It's maintainer-as-a-service.
Of course you don't have to hire me. But it is worth thinking about what needs to be done, and disaggregating it and seeing what bits companies can pay for, to help sustain the free software ecology they depend on.
So: sad, skill, sustain. I hope thinking about what approach you are taking helps your project think about maintainership, and what it needs to do to make the biggest long-term impact on software freedom. Thank you.
# 23 Feb 2016, 04:00PM: Leadership Crisis at the Wikimedia Foundation:
This week, the Wikimedia Foundation, the main organization supporting Wikipedia and several other free knowledge projects, is at the peak of a leadership crisis more than a year in the making. Molly White's timeline of the crisis is a useful guide to the facts, and I feel compelled to speak publicly for the first time about the problem, and share my personal perspective, with a bit of context to help non-Wikimedians understand.
I left the Wikimedia Foundation in September 2014 after four years. I mentioned the reasons then, in that post, around learning new things and working on projects with less of a public spotlight. I'm happy with my new direction, Changeset Consulting LLC, but still have so many fond memories of working with fantastic people and making a difference.
I left WMF thinking that it was fine -- in fact, that's a reason I felt okay about deciding to leave a place I cared about so much, because I thought WMF could cope without me. As I perceived it, former Executive Director (nonprofit-speak for "CEO" basically) Sue Gardner had led the organization to a stable enough place that she felt free to move on. For years, when I was at Wikimedia Foundation, our top priority was reversing the decline in the number of active Wikipedia contributors and other Wikimedia contributors; Sue Gardner articulated this priority and ensured everyone knew what we were aiming at and why. Lila Tretikov, the new executive director, was settling in and I perceived WMF to be on the right track, iteratively moving closer to reversing the editor decline, with solid management and plans in place to keep positive momentum going. I thought the conflicts and stumbles from summer 2014 were normal temporary pains, not unusually worrying.
A few months after I left, when I caught up with old Foundation colleagues, I started hearing wariness about the new high-level management (the ED and some other newer executive hires). The worries progressed into stronger and stronger concerns, getting more and more disturbing. For instance, in November 2015, a committee that disseminates Wikimedia funds to budget other Wikimedia institutions (such as chapters) wrote a scathing critique:
...the [Funds Dissemination Committee] laments that the Wikimedia Foundation's own planning process does not meet the minimum standards of transparency and planning detail that it requires of affiliates applying for its own Annual Plan Grant (APG) process....
The FDC is appalled by the closed way that the WMF has undertaken both strategic and annual planning, and the WMF's approach to budget transparency (or lack thereof).
I nearly could not believe my eyes when I saw this. For those of you who don't follow these bureaucracies, let me assure you that the FDC does not throw around words like "appalled" lightly. (Followup on the FDC recommendations.)
Early this year, it became public knowledge that the conflicts within the Foundation had led to an employee survey with a 93% response rate. The results included:
I have confidence in senior leadership at Wikimedia: 10% agree
It is a miracle if 90% of Wikimedia Foundation personnel agree on anything beyond the fact that WMF's commitment is to "a world in which every single human being can freely share in the sum of all knowledge." And to be clear, "confidence in senior leadership" here means that the employees trust that the C-level executives have the basic competence to run the organization. This isn't about agreeing or disagreeing on particular choices of method; when I was at WMF, the Executive Director and the Chief ____ Officers made decisions that some employees disagreed with, but they explained their reasoning, they encouraged feedback and responded to it (example), and we fundamentally knew they were aiming to collaborate with us in achieving the mission. It sounds like some big pieces of that trust are now missing.
1. In late December the Board of Trustees dismissed a well-liked community-elected trustee, Dr. James Heilman, for reasons that remain somewhat mysterious...
3. Revelations about newly appointed Board trustee Arnnon Geshuri's involvement in an illegal anti-poaching scheme while at Google has drawn community outcry
4. Besides failing to vet Geshuri, the WMF's increasing tilt toward the Silicon Valley and focus on (perhaps) the wrong technology projects have come into sharper relief
So, Arnnon Geshuri. You know that scandal where it came to light that big Silicon Valley tech firms were colluding to suppress wages and reduce employee mobility with illegal "no-poaching" agreements? With evidence including super damning emails? Guess who sent some of that email, perpetuating that pact? Arnnon Geshuri. The WMF Board of Trustees appointed him as one of the trustees. He's since stepped down but the incident damaged already-shaky trust between the Board and the larger Wikimedia community.
And as of last week it's clear that the situation's gotten even more dire. Fantastic colleagues are voting with their feet and leaving (and do you know how hard it is to find and hire the right people for an org this weird and this important?). People who would rather walk with rocks in their shoes than throw their coworkers under the bus are compelled to speak in public about dysfunction at the top: Ori Livneh, Anna Stillwell, and Greg Grossmeier, for instance, and Brion Vibber, who was the first employee the Foundation ever hired. Faidon Liambotis, Principal Operations Engineer at the Foundation & a longtime Debian Developer. Gayle Karen Young, former WMF Chief Talent & Culture Officer, who has a world-class ability to fuse compassion and systems-level thinking in the management of people processes, writes publicly about "dysfunction at the top" and "the enormous toll" it's taking on the staff. Erik Möller, who served on the Board of Trustees before he served as WMF second-in-command for more than seven years and then left in April 2015, a guy who has seen a thousand Wikimedia thunderstorms come and go and could probably charge for calm-as-a-service, says that "the situation is very much out of control" and "this is a crisis". This is not just the ordinary grumblings of a transparent organization. This is dire.
Executive Director Lila Tretikov said on Monday that Wikimedia has now managed to stem the editor decline. Möller replies and asks: is that so? and reviews the current stats, which do not reflect this claim.
But overall, it seems premature of speaking of "stemming the decline", unless I'm missing something (entirely possible).
There have been a thousand thunderstorms before. The Image Filter, SOPA, the transition from Toolserver to Tool Labs, Narrowing Focus, paid editing, the first VisualEditor rollout, the India Education Program, I could keep going and the point is that we Wikimedians have big ideas and big passionate arguments and we know some things about how to get through them. The movement, thank goodness, is bigger than the Foundation. The volunteers, the chapter staff, the teachers and photographers and coders and editors and everyone in the hundreds of subcommunities in our ecology have some buffers against the ripples coming out of the Foundation. There's frustration, sometimes, in how hard it can be for one subcommunity or organization to persuade or lend a hand to another, but right now it's a good thing because there is short-term resilience in that loosely federated structure.
This has been a singularly destructive time, but we still have time to keep the leadership problem from further damaging the Wikimedia movement.
As I said back in September 2014,
One of the things I admire about Wikimedia's best institutions is our willingness to reflect and reinvent when things are not working.
I don't know what the answer is.
The choice between exit and voice is conditioned on loyalty. We know that Wikimedians have been exercising the "voice" option; the Board and the ED have heard these criticisms loud and clear. And we know they've witnessed the stream of talented employees exiting, their steel-clad loyalty finally succumbing to the pressure. If unaltered, this is the kind of dynamic that leads to schisms and forks. I would hate for the movement to have to pay that kind of cost but, unless the Wikimedia Foundation Board of Trustees and Executive Director change course, I think that's a potential outcome.
# 19 Feb 2016, 06:50PM: What Should We Stop Doing? (FLOSS Community Metrics Meeting keynote):
"What should we stop doing?": written version of a keynote address by Sumana Harihareswara, delivered at the FLOSS Community Metrics Meeting just before FOSDEM, 29 January 2016 in Brussels, Belgium. Slide deck is a 14-page PDF. Video is available. The notes I used when I delivered the talk were quite skeletal, so the talk I delivered varied substantially on the sentence level, but covered all the same points.
I'd like to start with a story, about my excellent boss I worked for when I was at the Wikimedia Foundation, Rob Lanphier, and what he told me when I'd been on the job about eight months. In one of our one-on-one meetings, I mentioned to him that I felt overwhelmed. And first, he told me that I'd been on the job less than a year, and it takes a year to ramp up fully in that job, so I shouldn't be too worried. And then he reminded me that we were in an amazing position, that we would hear and get all kinds of great ideas, but that in order to get anything done, we would have to focus. We'd have to learn to say, "That's a great idea, and we're not doing it." And say it often. And, he reminded me, I felt overwhelmed because I actually had the power to make choices, about what I did with my time, that would affect a lot of people. I was not just cog # 15,000 doing a super specialized task at Apple.
So today I want to talk with you about how to use the power you have, in your open source projects and organizations, and about saying no to a lot of things, so you can focus on doing fewer things well -- the Unix philosophy, right? I'll talk about a few tools and leave you with some questions.
Tool 1: Remember to say no to the lamppost fallacy
The lamppost fallacy is an old one, and the story goes that a drunk guy says, "I dropped my keys, will you help me look for them?" "OK, sure. Where'd you drop them?" "Under that tree." "So why are you looking for them under this lamppost?" "Well, the light is better here."
A. Quantitative vs qualitative in the dev data
The first place we ought to check for the lamppost fallacy is in overvaluing quantitative metrics over qualitative analysis when looking at developer workflow and experience. Dave Neary said, in the FLOSSMetrics meeting in 2014, in "What you measure is what you get. Stories of metrics gone wrong": Use qualitative and quantitative analysis to interpret metrics.
When it comes to developer experience, you can be analytical while both quantitative and qualitative. And you rather have to be, because as soon as you start uncovering numbers, you start asking why they are what they are and what could be done to change that, and that's where the qualitative analytical approach comes in.
Qualitative is still analytical! Camille Fournier's post, "Qualitative or quantitative but always analytical", goes into this:
qualitative is still analytical. You may not be able to use data-driven reasoning because you're starting something new, and there are no numbers. It is hard to do quantitative analysis without data, and new things only have secondary data about potential and markets, they do not have primary data about the actual user engagement with the unbuilt product that you can measure. Furthermore, even when the thing is released, you probably have nothing but "small" data for a while. If you only have a thousand people engaging with something, it is hard to do interesting and statistically significant A/B tests unless you change things drastically and cause massive behavioral changes.
This is applicable to developer experience as well!
For help, I recommend the Wikimedia movement's Grants Evaluation & Learning team's table discussing quantitative and qualitative approaches you can take: ethnography, case studies, participant observation, and so on. To deepen understanding. It's complementary with the quantitative side, which is about generalizing findings.
B. Quantifiable dev artifacts-and-process data versus data about everything else
Another place to check for the lamppost fallacy is in overvaluing quantifiable data about programming artifacts and process over all sorts of data about everything else that matters about your project. Earlier today, Jesus González-Barahona mentioned the many communities -- dev, contributor, user, larger ecosystem -- that you might want to research. There's lots of easily quantifiable data about development, yes, but what is actually important to your project? Dev, user, sysadmin, larger ecology -- all of these might be, honestly, more important to the success of your mission. And we also know some things about how to get better at getting user data.
For help, I recommend the Simply Secure guides on doing qualitative UX research, such as seeing how users are using your product/application. And I recommend you read existing research on software engineering, like the findings in Making Software: What Really Works and Why We Believe It, the O'Reilly book edited by Andy Oram and Greg Wilson.
Tool 2: know what kind of assessment you're trying to do and how it plays into your theory of change
Another really important tool that will help you say no to some things and yes to others is knowing what kind of assessment you're trying to make, and how that plays into your hypothesis, your theory of change.
I'm going to mess this up compared to a serious education researcher, but it's worth knowing the basics of the difference between formative and summative assessments.
Formative assessment or evaluation is diagnostic, and you should use it iteratively to make better decisions to help students learn with better instruction & processes.
Summative assessment is checking outcomes at the conclusion of an exercise or a course, often for accountability, and judging the worth/value of that educational intervention. In our context as open source community managers, this often means that this data is used to persuade bosses & community that we're doing a good job or that someone else is doing a bad job.
As Dawn Foster last year said in her "Your Metrics Strategy" speech at the FLOSSMetrics meeting:
METRICS ARE USEFUL Measure progress, spot trends and recognize contributors.
Start with goals: WHY FOCUS ON GOALS? Avoid a mess: measure the right things, encourage good behavior.
Here's Ioana Chiorean, FLOSS Community Metrics meeting, January 30th 2015, "How metrics motivate":
Measure the right things... specific goals that will contribute to your organization's success
Dave Neary in 2014 in "What you measure is what you get. Stories of metrics gone wrong" at the Metrics meeting said:
be careful what you measure: metrics create incentives
Focus on business and community's success measurements
And this is tough. Because it can be hard to really make a space for truly formative assessment, especially if you are doing everything transparently, because as soon as you gather and publish any data, people will use it to argue that we ought to make drastic changes, not just iterative changes. But it might help to remember what you are truly aiming at, what kind of evaluation you really mean to be doing.
And it helps a lot to know your Theory of Change. You have an assessment of the way the world is, a vision of how you want the world to look, and a hypothesis about some change you could make, an activity or intervention you could perform to move us closer from A to B.
There's a chicken and egg problem here. How do you form the hypothesis without doing some initial measurement? And my perhaps subversive answer is, use ideas from other communities and research to create a hypothesis, and then set up some experiments to check it. Or go with your gut, your instinct about what the hypothesis is, and be ready to discard it if the data does not bear it out.
For help: Check out educational psychology, such as cognitive apprenticeship theory - Mel Chua's presentation here gives you the basics. You might also check out the Program/Grant Learning & Evaluation findings from Wikimedia, and try out how the "pirate metrics" funnel -- Acquisition, Activation, Retention, Referral, Revenue, or AARRR -- fits with your community's needs and bottlenecks.
Tool 3: if something doesn't work, acknowledge it
And the third tool is that when we see data saying that something does not work, we need to have the courage to acknowledge what the data is saying. You can move the goalposts, or you can say no and cause some temporary pain. We have to be willing to take bug reports.
Here's an example. The Wikimedia movement likes to host editathons, where a bunch of people get together and learn to edit Wikipedia together. We hoped that would be a way to train and retain new editors. But Wikipedia editathons don't produce new long-term editors. We learned:
About 52% of participants identified as new users made at least one edit one month after their event, but the percentage editing dropped to 15% in the sixth months after their event
And, in "What we learned from the English Wikipedia new editor pilot in the Philippines":
Inviting contribution by surfacing geo-targeted article stubs was not enough to motivate or help users to make their first edits to an article. Together, all new editors who joined made only six edits in total to the article space during this experiment, and they made no edits to the articles we suggested.
Providing suggestions via links to places users might go for help did not appear to sufficiently support or motivate these new editors to get involved. 50 percent of those surveyed later said they didn’t look for help pages. Those who did view help pages nevertheless did not edit the suggested articles.
But over and over in the Wikimedia movement I see that we keep hosting those one-off editathons. And they do work to, for instance, add new high-quality content about the topics they focus on, and some people really like them as parties and morale boosters, and I've heard the argument that they at least get a lot of people through that first step, of creating an account and making their first edit. But that does not mean that they're things we should be spending time on, to reverse the editor decline trend. We need to be honest about that.
It can be hard to give up things we like doing, things we think are good ideas and that ought to work. As an example: I am very much in favor of mentorship and apprenticeship programs in open source, like Google Summer of Code and Outreachy. Recently some researchers, Adriaan Labuschagne and Reid Holmes, raised questions about mentorship programs in "Do Onboarding Programs Work?", published in 2015, about whether these kinds of mentorship programs move the needle enough in the long run, to bring new contributors in. It's not conclusive, but there are questions. And I need to pay attention to that kind of research and be willing to change my recommendations based on what actually works.
We can run into cognitive dissonance if we realize that we did something that wasn't actually effective. Why did I do this thing? why did we do this thing? There's an urge to rationalize it. The Wikimedia FailFest & Learning Pattern hackathon 2015 recommends that we try framing our stories about our past mistakes to avoid that temptation.
Big 'F' failure framing:
- We planned this thing: __________________________
- This is how we knew it wasn't working: __________________________
- There might have been some issues with our assumption that: __________________________
- If we tried it again, we might change: __________________________
Little 'f' failure framing:
- We planned this thing: __________________________
- This is how we knew it wasn't working: __________________________
- We think that this went wrong: __________________________
- Here is how to fix it: __________________________
For help with this tool, I suggest reading existing research evaluating what works in FLOSS and open culture, like "Measuring Engagement: Recommendations from Audit and Analytics" by David Eaves, Adam Lofting, Pierros Papadeas, Peter Loewen of Mozilla.
I have a much larger question to leave you with.
One trend I see underlying a big chunk of FLOSS metrics work is the desire to automate the emotional labor involved in maintainership, like figuring out how our fellow contributors are doing, making choices about where to spend mentorship time, and tracking a community's emotional tenor. But is that appropriate? What if we switched our assumptions around and used our metrics to figure out what we're spending time on more generally, and tried to find low-value programming work we could stop doing? What tools would support this, and what scenarios could play out?
This is a huge question and I have barely scratched the surface, but I would love to hear your thoughts. Thank you.
Sumana Harihareswara, Changeset Consulting
# 19 Feb 2016, 06:10PM: Comparing Codes of Conduct to Copyleft Licenses (My FOSDEM Speech):
"Comparing codes of conduct to copyleft licenses": written notes for a talk by Sumana Harihareswara, delivered in the Legal and Policy Issues DevRoom at FOSDEM, 31 January 2016 in Brussels, Belgium. Video recording available. Condensed notes available at Anjana Sofia Vakil's blog.
Good afternoon. I'm Sumana Harihareswara, and I represent myself, and my firm Changeset Consulting. I'm here to discuss some things we can learn from comparing antiharassment policies, or community codes of conduct, to copyleft software licenses such as the GPL. I'll be laying out some major similarities and differences, especially delving into how these different approaches give us insight about common community attitudes and assumptions. And I'll lay out some lessons we can apply as we consider and advocate various sides of these issues, and potentially to apply to some other topics within free and open source software as well.
My notes will all be available online after this, so you don't have to scramble to write down my brilliant insights, or, more likely, links. And I don't have any slides. If you really need slides, I'm sorry, and if you're like, YES! then just bask in the next twenty-five minutes.
I will briefly mention my credentials in speaking about this topic, especially since this is my first FOSDEM and many of you don't know me. I have been a participant in free and open source software communities since the late 1990s. I'm the past community manager for MediaWiki, and while at the Wikimedia Foundation, I proposed and implemented our code of conduct, which we call a Friendly Space Policy, for in-person Wikimedia technical spaces such as hackathons and conferences.
I wrote an essay about this topic last year, as a guest post on the social sciences group blog Crooked Timber, and received many thoughtful comments, some of which I'll be citing in this talk.
I am also a contributor to several GPL'd pieces of code, such as MediaWiki and GNU Mailman, on code and non-code levels. And I am the creator of Randomized Dystopia, a GPL'd web application that helps you in case you want to write scifi novels about new dystopian tyrannies that abrogate different rights.
And I have been flamed for suggesting codes of conduct; for instance, one Crooked Timber commenter called me "a wannabe politician, trying to find a way to become important by peddling solutions to non-problems." Which is not as bad as when one person replied to me on a public mailing list and said, "Deja Vue all over again. I finally understand why mankind has been plagued by war throughout its entire history...." So maybe I'm the cause of all wars in human history. But I probably won't be able to cover that today.
II. The basic comparison
So let's start with a basic "theory of change" lens. When you're an activist trying to make change in the world, whether it's via a boycott, a new app, a training session, founding an organization, or some other approach, you have a theory of change, whether it's explicit or implicit. You have an assessment of the way the world is, a vision of how you want the world to look, and a hypothesis about some change you could make, an activity or intervention you could perform to move us closer from A to B. There's a pretty common theory of change among copyleft advocates and a couple of theories of change that are common to code of conduct advocates.
The GPL restricts some software developers' freedom (around redistributing software and around adding code under an incompatible license) so as to protect all users' freedom to use, inspect, modify, and hack on software.
The copyleft theory of change supposes that more people will be more free if we can see, modify, and share the source code to software we depend on, and so it's worth it to prohibit enclosure-style private takeovers of formerly shared code. Because in the long run, this will enable free software developers to build on each others' work, and incentivize other developers to choose to make their software free.
B. Codes of Conduct
Now, codes of conduct, antiharassment policies, friendly space policies: They restrict some people's behavior and require certain kinds of contributions from beneficiaries, so as to increase everyone's capabilities and freedom in the long run.
One pretty popular theory of change goes like this: we will make better software and have a greater impact if more people, and more different kinds of people, find our communities more appealing to work in. One thing making an unpleasant environment and driving away contributors, especially contributors with perspectives that are underrepresented in our communities, is hurtful misbehavior in community spaces. So we'll make the trade and say that it's worth it to restrict some behavior, in order to make the environment better so more, and more varied, people can do work in our communities, and thus make more free software and make it better.
And here's another related one, very similar to the one above, but focusing on the day-to-day freedom of community participants who are marginalized. If the constraint stopping me from, for instance, speaking in an IRC channel is that I strongly suspect I'll be harassed if they know I'm a woman, and that I don't have any reason to believe I can avoid or usefully complain about that harassment, how free am I to participate in that community? Is there perhaps a way to understand a certain level of safety as a necessary prerequisite to liberty?
I realize that this is probably the one room in the world where I have the highest chance of getting into a multi-hour "what does freedom mean" bikeshedding session, so I'm going to avoid focusing on the second model there and focus more on the first one, which emphasizes the end result of more free software.
So I am not assuming that everyone in this room is a copyleft advocate, but I am going to assume from this point forward that we in this room fundamentally understand the restrictive license argument, that we have a handle on the theory of change that it's operating on. And similarly, I'm sure there are people here who aren't so big on codes of conduct, but I'm going to assume that we fundamentally understand the theory of change behind that approach, regardless.
Now let's talk about similarities. Chris Webber calls both of these approaches "added process which define (and provide enforcement mechanisms for) doing the right thing." I agree. Without this kind of gatekeeping we see free rider incentives, on other people's software work and on other people's attention and patience and emotional labor.
They are written-down formalizations of practices and values that some community members think should be so intuitive and obvious that asking people to formally offer or accept the contract is an insult, or at least an unnecessary inconvenience. And so some people counterpropose sort-of-humorous policies, such as the "Do What the Fuck You Want to" software license and "don't be a jerk" codes of conduct.
They are loci of debate and fragmentation.
Some people agree to them thoughtfully, some agree distractedly as they would to corporate clickthrough EULAs, some disagree but click through anyway (acting in bad faith), some disagree and silently leave, some disagree and negotiate publicly, some disagree and fork publicly. Some people won't show up if the agreement is mandatory; some people won't show up UNLESS it's mandatory; some people don't care either way. And, by the way, good community management requires properly predicting the proportions, and navigating accordingly.
Both copyleft licenses and codes of conduct are approaches to solving problems that became more apparent along with different people realizing they have different expectations and needs, and consider different outcomes or processes to be "fair."
These kinds of codes and licenses usually cover specific bounded events and spaces or sites, and their scope covers interpersonal or public interactions. Codes of conduct usually don't cover conversations outside community-run spaces or the beliefs you hold in your head; open source licenses' restrictions usually kick in on redistribution, not use, so they don't constrain anything you do only on your own computer.
Neither one of these approaches can rely on self-enforcement. There is some self-enforcement of both, of course. There's a perception that -- as Harald K. commented on my blog post -- "licenses more or less police themselves (or in extreme instances, are policed by outsiders) whereas codes of conduct need an internal governing structure, a new arena where political power can be exercised." My personal understanding, which I share with people like Matthew Garrett, is that there's a ton of license-breaking happening, and we need to support existing organizations like the Software Freedom Conservancy to police that misbehavior and litigate to defend the GPL. As Conservancy head Karen Sandler points out in her December essay "From a lawyer who hates litigation", "I've seen companies abuse rights granted to them under the GPL over and over again. As the years pass, it seems that more and more of them want to walk as close to the edge of infringement as they can, and some flagrantly adopt a catch-me-if-you-can attitude." And you see enough individuals in our communities acting similarly that I don't think I need to belabor this point; codes of conduct are much more productive when they're actually, you know, enforced.
And both copyleft licenses and codes of conduct restrict freedom regarding certain acts, over and above what is restricted by the law, in the interest of a long-term good, which can in both cases be construed as greater freedom. As Belle Waring says to one skeptic in the Crooked Timber comments, paraphrasing their argument: "part of your reasonable resentment is, 'I don't want to be forced to do freedom-restricting things in support of a very uncertain outcome, just because the final proposed outcome is a good one.'" I will go into that bit of argument later.
But these kinds of agreements are different on a few different axes, which I think are worth considering for what they tell us about open stuff community values and about our intuitions on what kinds of freedom restrictions we find easier to accept.
One is that many codes of conduct focus on in-person events such as conferences, rather than online interactions. Many of the unpleasant incidents that caused communities to adopt CoCs -- or that communities see as "let's not let that happen here" warning bells -- happen at face-to-face events. And face-to-face spaces have a much longer history and context of ways of dealing with bad behavior than do online spaces. After all, a pretty widespread reading of the core function of government and law enforcement is that they keep Us Good Guys safe by stopping The Bad Guys from committing face-to-face (or knife-to-face or chair-to-face) assault.
But there's another axis I want to explore here: whether the behavior constraint feels like a contract or whether it feels like governance. Of course, we toss around phrases like "the social contract" and use the metaphor of contract to talk about the legitimacy of government, but to an ordinary citizen, contracts and governance feel like significantly different things. To oversimplify: to a non-lawyer like me, something that feels like a contract formalizes a specific trade, something discrete and finite and a bit rare. A copyleft license feels that way to me; it specifies that if I distribute a certain artifact -- which is something I would only do after some amount of thought and work -- I then also undertake certain obligations, namely, I must also redistribute the software's source code, under the same license. And, notwithstanding edge cases, it is often easy to examine the artifact, follow a decision procedure, and determine that I have complied with the terms of the license. If I meant to comply in the first place.
On the other hand, when we make rules constraining acts, especially speech acts, it feels more like governance.
Codes of conduct serve as part of a community's infrastructure to fulfill the first duty of a government — to protect its citizens from harm — and in order to make them work, communities must develop governance processes. That is to say, "governance" is what we call it when we're explicit about who gets to make and implement rules that affect everyone in a community, and how we choose those people or get rid of them. And a governance body does not necessarily have to be a legal entity. For instance, in MediaWiki governance, there's an architecture committee that decides on large technical architectural changes, and it has no standing in the eyes of the United States government.
It takes work to evaluate whether actions have complied with rules, and that work might require asking questions of suspects, bystanders, and targets. Enforcing a code of conduct, even a narrowly scoped anti-harassment policy, often requires that someone act on behalf of a community to do this, and to implement the outcome -- be it informed by retributive, rehabilitative, transformative, or some other justice model. And it feels more like governance than contract to me if a rule applies to actions I take many times a day without deliberate planning -- such as saying something in my project's live internet chat room.
One way of thinking about this is: is there some kind of authority that the community acknowledges as having legitimate power over everyday behavior, over and above existing government with a capital G? Because, again, licenses affect certain coding and architectural decisions, but they don't preclude, for instance, everyday discussion. In fact, the social and digital infrastructure it takes to make robust and usable software, including our bug reports, our automated tests, our conversation on mailing lists, and so on, is often not covered by any particular open license -- if it were, maybe we'd be seeing a different level of pushback even from developers who are happy with copyleft as applied to their code.
F. Shortcomings of the contract model
But I think another interesting thing that happens when you compare a governance model to a contract model, regarding approaches we take to improving behavior in our communities, is seeing how governance wins. It takes a lot of work, but it has a lot of advantages.
Contracts are binary where ongoing dialogue and governance can be more flexible and responsive. If I were going to be really annoying I would compare them to compiled bytecode and to interpretable scripts. Contracts have to sort of self-contain the tests for what the contract permits, mandates, and prohibits, whereas governance mechanisms and bodies can use more general standards, which might change over time. To quote one of the commenters on my essay, Stephenson-quoter kun:
contracts explicitly restrict acts which are simply unpardonable -- not sharing the source code to your modified version of a GPL-licensed project, sexually assaulting someone at a conference -- because everyone agrees that those things are wrong and we feel confident that we can agree up-front that there can never be any extenuating circumstances in which those things are actually OK. Governance, however, can serve to 'nudge' people away from bad behaviours – poor coding standards, rudeness on mailing lists -- by giving us a standard to measure those things against without enumerating every possible violation of the standard. A governance procedure can take context into account, and is much more easily subject to improvement and revision than a contract is.
Sometimes it's the little stuff, more subtle than the booth babe/groping/assault/slur kind of stuff, that makes a community feel inhospitable to me. When I say "little stuff" I am trying to describe the small ways people marginalize each other: dominance displays, cruelty in the guise of honesty, the use of power in inhospitable ways, feeling unvalued, "jokes", clubbiness, watching my every public action for ungenerous interpretation, nitpicking, and bad faith.
Changing these habits requires a change of culture, and that kind of deliberate change in culture requires people who take up the responsibility in stewarding the culture.
And a governance approach has a lot more ability to affect culture than a contracts-only approach does.
2. Contracts give us an illusion of equality and self-containedness
As Tim McGovern said in the comments to my Crooked Timber post:
contracts have taken over as a primary way of negotiating relationships: a EULA is a replacement for a legal understanding of the relationship between two parties who are doing business. I don't, in other words, sign a EULA when I buy a pair of socks -- or even when I buy a car (Teslas excepted) because the purchase relationship is legally defined; even the followup on what can and can't be in your warranty is legally defined. But companies would rather be bound by an agreement they write than a body of law based on either commonlaw or constitutional concepts, or legislation.
Contracts presume an equality between the parties; in theory, both sides can take a breach of contract to court. In practice, of course, a EULA is a contract that masks radical inequality in power between the parties.... Governance requires wrestling with equality in a real way, on the other hand, and voluntarily submitting to an authority constituted in some fashion (over time, by people, etc.), as opposed to preserving a contractual illusion of equality.
3. Contract pretends you have choices
I recommend that, if you haven't, you check out the article "Mothering versus Contract" by Virginia Held, from Beyond Self-Interest in 1990. It suggests that perhaps we should fundamentally conceive of our interactions with others as following a paradigm of motherhood rather than of contract -- one truth this approach acknowledges is that by default most interactions in your life are opt-out rather than opt-in, if there's any opting or choice at all.
Yes, there's the freedom to fork. But realistically, if you want to get things done, you have to collaborate with others, and we need to accede to other people's demands, in terms of interface compatibility, learning and speaking fluent English, and all sorts of other needs. A FLOSS project with a thriving ecology of contributors is far more valuable than a nearly identical chunk of code with only a couple of voices available to help out, and thus the finite amount of human attention limits our ability to make effective forks. We're more inderdependent than independent, and acknowleding that as a fundamental truth complicates the contracts-y libertarian narrative potentially beyond usefulness.
I hope that my analysis helps give some vocabulary and frameworks for understanding arguments around these issues, and that we can use them to develop more effective arguments.
A. Freedom tradeoff comparisons
The first step might be — if you're trying to get your community to adopt a code of conduct, you might benefit by looking at other freedom-restricting tradeoffs the community is okay with, so you can draw out that comparison.
Or with UX (user experience) -- design is the art of taking things away, and when you're advocating for better user experience, which often involves reducing the number of visible ways to do things, consider comparing your approach to one of the freedom tradeoffs that your interlocutor is already okay with, such as the fact that your community has standardized on a single version control system. A single way for that kind of user to interact.
And if you're trying to build a code of conduct consensus in your community, it might help to start by talking, not about day-to-day beavior, but about artifacts that people think of as artifacts. Talk about the things we make, like slide decks for presentations, articles on your wiki. That can get people on the same page as you, in case they're not yet ready to think of the community itself as an artifact we make together.
C. Theory of change
If you're an advocate for a new initiative, licensing, code of conduct, or something else, understand your own theory of change, and build mental models to help you understand the people who disagree with you. Understand what part of the theory of change they disagree with, and gather data to counter it.
And, incidentally, this lens will also help you appreciate other complementary approaches that will help you achieve your goals. As Mike Linksvayer says: "Of course I think that copyleft advocates who really want to ensure people have software freedom rather than just being enamored of a hack should be always on the lookout for cheaper and/or socialized enforcement (as implied above, control of distribution channels that matter, and state regulation)."
So why might people oppose codes of conduct? Here are a few ideas:
- they might disagree on whether the goal makes sense
- or on whether codes of conduct, when enforced, make the situation more conducive to diverse populations and to net growth in community -- have your research close at hand!
- or on what the biggest problems you're facing are, and whether they're community recruitment and retention
As Chris Webber notes, "there's an argument that achieving real world social justice involves a certain amount of process, laying the ground for what's permitted and isn't, and (if you have to, but hopefully you don't) a specified direction for requiring compliance with that correct behavior." The addendum is that, as Alberto Brandolini said "The amount of energy necessary to refute bullshit is an order of magnitude bigger than to produce it."
So part of the mental model you're trying to understand is what the person you're arguing with is trying to maximize, and another part is whether you agree on how to maximize it.
Paul Davis, the Ardour BDFL, commented on my Crooked Timber post, "The dilemma for a mid-size project like mine is that the overhead of developing and maintaining a CoC seems like just another thing to do amidst a list of things that is already way too long, and one that addresses a problem that we just don't have (yet)." He said he's more worried about technical, architectural decisions causing developer loss.
So, for instance, you could argue with Paul: what genuinely causes developer loss? And what priorities should you have, given your goals?
D. A fresh set of governance needs and questions
CoC adoption drives the adoption of explicit governance mechanisms, as Christie Koehler has recently explored in depth in her post "The complex reality of adopting a meaningful code of conduct" .... but we have many open questions that the legal and policy community within free and open source could really help with.
For instance, it's great that we have people like Ashe Dryden and organizations like Safety First PDX helping develop standards and advising organizers on developing and enforcing codes of conduct, but should we actually be centralizing this kind of reporting, codification and enforcement across the FLOSS ecosystem? Different subcommunities have different needs and standards, but just as OSI has helped us stave off the worst possibilities of license proliferation, maybe we should be avoiding the utter haphazardness of Code of Conduct proliferation.
And -- given how interconnected our projects are -- what if single open source projects are the wrong size or shape or scope for this particular aspect of stewardship and governance?
I'd very much appreciate thoughts on this from other folks in future devroom talks or blog posts -- if you tell me this is the kind of thing we talk about on the FLOSS Foundations mailing list then maybe I'll have to bite the bullet and go ahead and subscribe.
IV. Other thoughts + Conclusion
A. Comments on my CT piece
The comments on my Crooked Timber piece had many fine insights, on enforcement, culture, exit, voice and loyalty, fairness, and the consent of the governed. They're worth reading.
B. Hospitality to liberty spectrum
In addition to the contract-governance contrast, I think it's also worth thinking about the spectrum of liberty versus hospitality. The free software movement really privileges liberty, way over hospitality. And for many people in our movement, free speech, as John Scalzi put it, is the ability to be a dick in every possible circumstance. Criticize others in any words we like, and do anything that is not legally prohibited.
Hospitality, on the other hand, is thinking more about right speech, just speech, useful speech, and compassion. We only say and do things that help each other. The first responsibility of every citizen is to help each other achieve our goals, and make each other happy.
I think these two views exist on a spectrum, and we are way over to one side, the liberty side, as a community, and moving closer to the middle would help everyone learn better and would help us keep and grow our contributor base, and help make it more diverse. And to the extent that comparing codes of conduct to copyleft licenses helps some people put new initiatives in perspective, balancing the relationship between rights and responsibilities, perhaps that can also help shift our culture into one that's more willing to be hospitable. I hope.
C. This feels like a potentially insoluble problem
William Timberman said in Crooked Timber comments, "how does a socialist persuade a libertarian that coherence and the common good is sometimes a legitimate constraint on individual freedom?" And the answer is that I don't know, but I hope it is a soluble problem, and I hope I've opened up some avenues for exploration on that topic. Thank you.
# 26 Jan 2016, 02:42PM: Recent Discussion on Unfairness in FLOSS Economics:
I'm keenly watching the conversation on structural imbalances in funding and use of free and open source software. Nadia Eghbal's recent essay has garnered attention, and here I collect some additional posts and threads by others about this disparity in the economics of FLOSS:
- "Problems and Strategies in Financing Voluntary Free Software Projects" by Benjamin Mako Hill, originally published 2005, revised November 2012.
- "Seeing like a Geek" by Tom Slee, June 2012. "...the Open Data Movement demands that data ... be accessible to rich and poor alike, like justice and the Ritz. It insists that any measures governments would like to take to favour--for example--non-commercial users or local users, be taken off the table."
- "The Ethics of Unpaid Labor and the OSS Community" by Ashe Dryden, November 2013.
- "For Love and For Money" by Audrey Eschright, June 2015. "What if it this framework was always about money? The purpose of sanctioning open source licenses at all is to create a regulatory environment for commercial use of code."
- "Funding OSS" by Cory Benfield, August 2015.
- "The Predatory Search And Exploitation of Free Labor" by Cameron G., September 2015. "This is the truth that too many marginalized workers are now familiar with: the industries we know and love are being built on our free labor, our hunt for 'experience,' and our naivety about our worth."
- The paying the piper discussion forum on GitHub, started October 2015. Concentrating on the need to fund "full time, dedicated project management and contribution staff."
- "Corporations and OSS Do Not Mix" by Ian Cordasco, November 2015. "Companies need to have realistic expectations of the work-life balance of open source maintainers."
- "Relying on volunteering is more unfair than you think" by Sasha, November 2015.
- "Open Source Work" by Ryan Bigg, November 2015. "As of today I am quitting all maintenance roles / responsibilities of any open source project I am involved in."
- "The Acute Pain and Chronic Reward of Public-Facing Work" by Trey Causey, November 2015.
- "Funding free software" by Ned Batchelder, November 2015.
- Hypatia Software Organization ("Software engineer mentorships empowering transgender women") started, November 2015. December: shares estimates on revenue and distribution. [Update: Lillian Lemmer, the Executive Director of Hypatia Software Organization, reached out to me in February to share an updated Transparency link.]
- "How to help your favourite open source project" by David R. MacIver, December 2015.
- "Launching Jazzband" by Jannis Leidel, December 2015. "Jazzband is a cooperative experiment to reduce the stress of maintaining Open Source software projects."
- "Starters and Maintainers" by James Long, December 2015.
- "GPL enforcement is a social good" by Matthew Garrett, December 2015.
- dear GitHub (an open letter to GitHub from FLOSS maintainers), initiated by James Kyle, January 2016.
- No Maintenance Intended signal/initiative initiated by Potch, January 2016. "Tell people your code is open source, but not actively maintained."
- "Linux Foundation quietly drops community representation" by Matthew Garrett, January 2016.
- Crowdsourced FLOSS project list, started by Nadia Eghbal and improved by Jannis Gebauer, January 2016. Lists FLOSS projects in need of various kinds of support.
I include above some pieces that, on the surface, are adjacent to this conversation rather than in it: on open data, on emotional burnout, on GitHub's tooling, on license compliance, on setting expectations about unmaintained projects. But I see these frustrations as -- like the injustice driving volunteer maintainers to step away -- coming from a fundamental perception of unfairness. Free and open source software makers will notice if there is no measure of reciprocity in an environment that pays lip service to gift culture.
My next step probably ought to be reading the work of Nobel Prize winner Elinor Ostrom: "groundbreaking research demonstrating that ordinary people are capable of creating rules and institutions that allow for the sustainable and equitable management of shared resources." I do hope so.
# 21 Jan 2016, 11:00AM: Risk Mitigation:
Next week I'm headed to Belgium for my first Free and Open Source Developers' European Meeting. I'll give two talks. I'm excited, because it'll be a chance to listen, learn, influence, introduce myself to potential clients, and see old pals.
But I asked one old pal whether he'd be there and got the reply:
Don't plan to be at FOSDEM; one of these years, maybe after their CoC isn't a joke.
For some time, FOSDEM participants and people who'd like to attend have asked FOSDEM organizers to improve their Code of Conduct. In October, one of the people organizing the Legal and Policy Issues DevRoom suggested,
FOSDEM is a fantastic conference and the only thing I can think of that would make it better is publishing a Code of Conduct...
Discussion ensued, and in November, the organizers announced their new Code of Conduct. I appreciate that different organizations need to customize their anti-harassment/friendly space/conduct policies, as the Wikimedia technical community did under my leadership, and I recognize that FOSDEM -- entirely volunteer-run, requiring no attendee registration, and charging no admission fee -- has its own particular challenges. But I see why my friend looks askance at FOSDEM's CoC. If you compare it to the example policy offered on the Geek Feminism wiki, you see how lots of little differences add up. For instance, FOSDEM's policy doesn't give a way to anonymously report a problem, and it doesn't suggest how you can find or identify team or staff members.
I figure I can go, this time, see how it goes, keep my guard up a bit, and then, as a member with more standing in and a more nuanced understanding of the FOSDEM community, ask for specific improvements, and explain why. My support network, my judgment, and my courage are in good enough shape that I can handle the most likely nonsense without taking too much damage.
But there's this one wrinkle.
The night before FOSDEM proper, the organizers run a beer night that -- according to my friends who have attended -- is a highlight of the convention. Since many FOSDEM attendees spend the session days in subject-specific devrooms, and since I want to meet people from many and varied projects, this beer night is probably the most high-value networking event all weekend. But. As the Geek Feminism wiki astutely notes,
Intoxication (usually drunkeness) both genuinely lowers inhibitions and provides people with an excuse for acting badly even if they genuinely knew better.
The data makes me cautious. FOSDEM improved its policy, but not enough to completely reassure me, and we still have yet to see how they implement it. Many individual devrooms and affiliated events, such as the FLOSS metrics meeting where I'm speaking, have added their own CoCs, but that doesn't cover the beer night.
So how will I mitigate risk? Maybe I won't go to the beer party at all. Maybe I'll go, but stay in loud crowded places, even if that makes it harder for me to have the kinds of in-depth conversations that lead to sales. Maybe I'll mention my husband a lot and dress androgynously. Maybe I'll mostly talk with women, with other nonwhite people, and with friends I already know, trading off serendipity against safety. And, despite the organizers' suggestion that I "don't miss this great opportunity to taste some of the finest beer in Belgium," and even though I enjoy trying new beers, I'll probably stick to water.
(And then next year I'll be part of the whisper network, helping other folks decide whether to go.)
I'm writing this to help people who don't have to make these risk calculations see a snapshot of that process, and, frankly, to justify my attendance to those who can't or won't attend FOSDEM till it's more clearly dedicated to a harassment-free experience for participants. And comments on this blog post are closed because, as Jessica Rose said:
Any extended conversation around a code of conduct will eventually demonstrate why a code of conduct is necessary.
P.S. I tried to think of an appropriate "free-as-in-beer" joke and could not. Regrets!
# 20 Jan 2016, 12:56PM: Several Upcoming Talks:
I'm preparing several talks to deliver at open source technology conferences this winter and spring.
I'll be at FOSDEM in Brussels later this month giving two talks:
- On Friday, January 29th, at the FLOSS Community Metrics Meeting, I'm presenting "What should we stop doing?" The FLOSS community often clamors for stats that would let us automate emotional labor, so we could focus on more valuable work. Is that appropriate? What if we switched our assumptions around and used our metrics to figure out what we're spending time on more generally, and tried to find low-value programming work we could stop doing? What tools would support this, and what scenarios could play out?
- On Sunday, January 31st, I'm speaking in the Legal and Policy Issues "devroom" on comparing codes of conduct to copyleft licenses, expanding on the discussion I started in this Crooked Timber piece last year. What can we learn about our own attitudes towards governance when we look at how and whether we make these different freedom tradeoffs?
In mid-March, I will present "Hidden Features in HTTP" at Great Wide Open in Atlanta, Georgia. This will be pretty similar to "HTTP Can Do That?!", which I presented to a standing-room-only crowd at Open Source Bridge last year. If you're a web developer whose knowledge of HTTP verbs ends around
POST, expect news, laughs, and lab reports from wacky experiments.
Right after Great Wide Open, I'll speak on "Inessential Weirdnesses in Free Software" at LibrePlanet in Boston. And then in mid-May, I'll be presenting "Inessential Weirdnesses in Open Source" at OSCON in Austin, Texas. More than a year after I wrote "Inessential Weirdnesses in Open Source" as a tossed-off blog post, I'm pretty dissatisfied with it. I should have more clearly stated my assumptions and audience, and my intent to play around with some vocabulary and what-ifs; I'm unhappy that many people misread it as a "we should eradicate all these things" manifesto. In these talks I aim to clarify and deepen this material. Open source contributors and leaders who are already comfortable with our norms and jargon will learn how to see their own phrasings and tools as outsiders do, including barriers that often slow down new users and contributors, and to make more hospitable experiences during their outreach efforts.
Then in late May I'll make a public appearance or two at WisCon -- the exact nature of which is a surprise!
I'm proud that this year I'll be speaking for the first time at FOSDEM, Great Wide Open, LibrePlanet, and OSCON. I hope my talks and the hallway track help me get the word out about Changeset Consulting to potential clients.
And if you can't make it to any of those conferences, but you'd like to hear more about Changeset and my other activities, check out Andromeda Yelton's one-hour interview with me in her Open Paren video podcast. At 39:29 I emit a huge belly laugh that makes me happy to re-watch and you might like it too.
# 28 Dec 2015, 02:06PM PST: What Software Freedom Conservancy Does, Why It's Important, And Why You Should Give:
I appreciate the work of the Software Freedom Conservancy, a nonprofit that helps free and open source software projects. Right now they need 2,500 people to become Supporters to keep their work going. So I made a video about why I support them, using language and examples that you can understand if you're new to this topic. It's embedded below, along with the text script I spoke from.
This month, I'm volunteering to help raise money for the Software Freedom Conservancy. My local bookshop does something cool for the holidays: volunteers wrap gifts for free, and any tips from the customers go to a charity that the volunteer gets to choose. So I've been explaining to the customers (most of whom aren't technologists) that I am donating their tips to the Software Freedom Conservancy.
My one-sentence explanation: The Software Freedom Conservancy is a nonprofit that helps programmers give away their software for free.
If they are curious, I explain further:
One way they do this is by being a nonprofit umbrella. Developers who want to make software and give it away often need a way to take donations and spend them on stuff like travel (to see each other and work face-to-face). Setting up their own nonprofits would take a ton of time and paperwork and filing fees. So the Conservancy takes care of all that, handling the accounting and stuff like that.
Another thing they do is license compliance work. You see, if you just write something, then automatically, the license that applies is standard copyright. But programmers who want to give away their software do it by saying it's under a different license, one that says, it's fine for you to copy this and look at the code and change it and even give it or sell it to other people, as long as you let other people do the same thing, too. But there are some companies that don't follow these rules. They maybe reuse these things that other people gave away, and package them into a phone or a tablet or something, and then they close it up. They don't let other people see that code -- they don't give other people the same chance that they benefited from. So the Conservancy follows up on that, sends them legal letters that say, "hey, that's illegal, that's not fair, don't do that."
And another thing they do is, there's this internship program, a paid internship program called Outreachy, to help get women and other underrepresented groups into this part of the tech industry. You see, most internships in the software industry are paid -- it's not like a lot of other industries. We gotta pay these interns to help them get into this part of the industry. So the Conservancy is the nonprofit umbrella for this program, and handles the finances so that companies can donate money and the interns can get paid.
That's my explanation. I'm glad I can help tell people about this great nonprofit and the unique work they do. And it really is unique. So if you or people you care about have benefited from the Conservancy's work, or if you just think it's a good idea, please give them $120, or whatever you can, during this fundraiser, and spread the word. Thank you.
Technologists might also like Matthew Garrett's "GPL enforcement is a social good" and Mike Linksvayer's thoughts on his favorite Conservancy accomplishment of 2015.
Please give -- right now, there's a match available that will make your gift count twice!
Edited 6 February to add: The donation match runs till 1 March 2016. Please give.
# 16 Sep 2015, 01:03PM: Software In Person:
In February, while coworking at the Open Internet Tools Project, I got to talking with Gus Andrews about face-to-face tech events. Specifically, when distributed people who make software together have a chance to get together in person, how can we best use that time? Gus took a bunch of notes on my thoughts, and gave me a copy.
Starting with those, I've written a piece that Model View Culture has published today: "Software In Person".
Distributed software-making organizations (companies, open source projects, etc.) generally make time to get people together, face-to-face. I know; I've organized or run hackathons, sprints, summits, and all-hands meetings for open source projects and businesses (and if I never have to worry about someone else's hotel or visa again, it'll be too soon).
Engineers often assume we don't need to explicitly structure that time together, or default to holding an unconference. This refusal to reflect on users' needs (in this case, the participants in the event) is lazy management. Or event organizers fall back to creating conferences like the ones we usually see in tech, where elite men give hour-long lectures, and most participants don't have any opportunities to collaborate or assess their skills. Still a bad user experience, and a waste of your precious in-person time.
Why do you think you're spending hundreds of thousands of dollars holding hackathons, sprint weeks, and conferences? And how could you be using that time and money better?
Subsections include "Our defaults", "Investing for the long term", "Beyond 'hack a lot'", "Grow your people", and "Setting yourself up for success". Thanks to Gus and to Model View Culture for helping me make this happen!
# 09 Aug 2015, 10:52PM: How To Improve Bus Factor In Your Open Source Project:
Someone in one of my communities was wondering whether we ought to build a new automated tool to give little tasks to newcomers and thus help them turn into future maintainers. I have edited my replies to him into the How To Build Bus Factor For Your Open Source Project explanation below.
In my experience (I was an open source community manager for several years and am deeply embedded in the community of people who do open source outreach), getting people into the funnel for your project as first-time contributors is a reasonably well-solved problem, i.e., we know what works. Showing up at OpenHatch events, making sure the bugs in the bug tracker are well-specified, setting up a "good for first-timers" task tag and/or webpage and keeping it updated, personally inviting people who have reported bugs to help you solve them, etc. If you can invest several months of one-on-one or two-on-one mentorship time, participate in Google Summer of Code and/or Outreachy internship programs. If you want to start with something that's quantitative and gamified, consider using Google Code-In as a scaffold to help you develop the rest of these practices.
You need to quickly thank and give useful feedback to people who are already contributing, even if that feedback will include criticism. A fast first review is key, and here's a study that backs that up. Slide 8: "Most significant barrier to engaging in onramping others is unclear communications and unfriendly community. Access to the right tools has some effect." Slide 26:
"Contributors who received code reviews within 48 hours on their first bug have an exceptionally high rate of returning and contributing. (And "Github, transparency, and the OTW Archive project" discusses how bad-to-nonexistent code review and bad release management led to a volunteer dropping out of a different open source project.)
Contributors who wait longer than 7 days for code review on their first bug have virtually zero percent likelihood of returning.
Showing a contributor the next bug they can work on dramatically improves the odds of contributing."
In my opinion, building bus factor for your project (growing new maintainers for the future) is also a solved problem, in that we know what works. You show up. You go to the unfashionable parts of our world where the cognitive surplus is -- community colleges, second- and third-tier four-year colleges, second- and third-tier tech hubs, boring enterprise companies. You review code and bug reports quickly, you think of every contributor (of any sort) as a potential co-maintainer, and you make friendly overtures to them and offer to mentor them. You follow OpenHatch's recommendations. You participate in Google Summer of Code and/or Outreachy internship programs.
Mentorship is a make-or-break step here. This is a key reason projects participate in internship programs like GSoC and Outreachy. For example, Angela Byron was a community college student who had never gotten involved in open source before, and then heard about GSoC. She thought "well it's an internship for students, it'll be okay if I make mistakes". That's how she got into Drupal. She's now a key Drupal maintainer.
Dreamwidth, an open source project, started with two maintainers. They specifically decided to make the hard decision to slow down on feature development, early on, and instead pay off technical debt and teach newcomers. Now they are a thriving, multimaintainer project. "dreamwidth as vindication of a few cherished theories" is perhaps one of my favorite pieces on how Dreamwidth did what it did. Also see "Teaching People to Fish" and this conference report.
Maintainers must review code, and that means that if you want someone to turn into a maintainer in your project, you must help them learn the skill of code review and you must help them get confident about vetoing and merging code. In my experience, yes, a good automated test suite does help people get more confident about merging changes in. But maintainers also need to teach candidates what their standards ought to be, and encourage them (many contributors' first thought when someone says "would you want to comaintain this project with me?" is "what? me? no! I'm not good enough!"). Here's a rough example training.
If you want more detailed ways to think about useful approaches and statistics, I recommend Mel Chua's intro to education psychology for hackers and several relevant chapters in Making Software: What Really Works and Why We Believe It, from O'Reilly, edited by Greg Wilson & Andy Oram. You'll be able to use OpenHub (formerly Ohloh) for basic stats/metrics on your open source project, including numbers of recent contributors. And if you want more statistics for your own project or for FLOSS in aggregate, the open source metrics working group would also be a good place to chat about this, to get a better sense of what's out there (in terms of dashboards and stats) and what's needed. (Since then: also see this post by Dawn Foster.)
We know how to do this. Open source projects that do it, that are patient with the human factor, do better, in the long run.
# 04 Aug 2015, 01:14AM: My Eulogy for Nóirín Plunkett:
A few hours ago, I spoke at Nóirín's memorial service. This is what I said (I am sure I varied the words a bit when I read it).
My name is Sumana Harihareswara, and I will always remember Nóirín's compassion, insight, and bravery.
They were brave to publicly name and fight back against wrongs done against them -- by members of the open source community -- wrongs done against them and others; I think it is not exaggerating to say that their bravery galvanized a movement. Our open technology community owes them a debt that can never be repaid.
We also benefited tremendously from their insight. Nóirín had just started a new role at Simply Secure, one that combined their expertise in open stuff with their writing and coordinating skills, and their judgment and perspective. And before that, when they worked as a project manager for the Ada Initiative, I had the privilege of working closely with Nóirín; I am grateful for that, but of course now I know what I'm missing, what we're all missing, because I had the chance to see, every day, their diligence and insight and discretion and judgment and empathy, and compassion. Some of us lead like engineers, by making systems that scale; some of us lead like nurturers, cultivating relationships and trust with emotional labor. Nóirín was brilliant at both of those, and I wish I could have decades more to learn from them, and toss around more ideas and frameworks.
The last time I saw Nóirín was at WisCon, a feminist science fiction convention in May. One morning I came down the hotel stairs and saw them seated against a wall, crying, sobbing, because Ireland had just passed a referendum legalizing same-sex marriage. They were so happy that their friends and loved ones and everyone back home were now freer to marry and have their families recognized that they'd gotten a glass of champagne from the hotel restaurant, at maybe eight in the morning, to celebrate. They felt deeply the joy and suffering of others.
Nóirín, I miss you, and I will try to live up to the example you set. Thank you.
# 03 Aug 2015, 09:33AM: Memorial Service Details:
A nondenominational memorial service for Nóirín Plunkett will be tonight, August 3, at 6:30pm, in Cambridge, Massachusetts (word had been publicized on Twitter and Identi.ca). All who knew Nóirín are invited. I will be there.
# 29 Jul 2015, 11:49AM PST: On Nóirín Trouble Plunkett's Death:
I was devastated today to learn of the death of my friend Nóirín Plunkett.
This is a terrible thing and I am still shocked and saddened to learn of their death. (Per their profile, please follow their pronoun preferences and use "they".)
Some things to know about them:
Their bold honesty about being sexually assaulted at an open source software event moved us to action; it helped spark the creation of the Ada Initiative.
As Geek Feminism's wiki documents, they were facing tremendous legal bills because of a legal conflict with an ex.
They had just started a new role at Simply Secure, one that combined their open tech expertise with their writing and coordinating skills and their judgment and perspective.
When I was volunteering on the search for the Ada Initiative's new Executive Director, I worked closely with Nóirín and could always count on their wisdom, compassion, and diligence. I am so grateful, now, that I had a chance to collaborate with them -- I had hoped to work with them again, someday, in some organization or other.
One of the last times I saw them, they were crying with happiness over the passage of the Irish same-sex marriage referendum.
I don't want to end this entry because there is no ending that can do justice to them.
# 18 Jun 2015, 06:53AM: HTTP Can Do That?! and Comedy:
On Wednesday of next week (June 24th) I'm presenting "HTTP Can Do That?!" at Open Source Bridge in Portland, Oregon.
I have explored weird corners of HTTP -- malformed requests that try to trick a site admin into clicking spam links in 404 logs, an API that responds to POST but not GET, and more. In this talk I'll walk you through those (using Python, netcat, and other tools you might have lying around the house).
I practiced this talk Tuesday night at the Recurse Center and it went well; people learned a lot about headers, verbs, status codes, and odd HTTP loopholes, and gave me constructive criticism so next week's version will be clearer.
I have also suggested a Birds of a Feather evening session called "Nothing Is Totally Incomprehensible If We Try Together" but don't yet know whether or when it will happen.
Then, at AlterConf Portland on Saturday, June 27th, I'll be performing some stand-up comedy for hippie nerds. I thought about trying to cram 100 punchlines into my 45-minute HTTP talk, but I don't think I'll be able to achieve that -- people need to understand something before they can understand a joke about it -- so it'll be nice to get 4 or 5 laughs per minute during the stand-up on Saturday.
# 23 May 2015, 10:41PM CST: New Vid: Pipeline:
I've made a new fanvid: "Pipeline". It's a little over 3 minutes long and cuts together about 50 different sources (documentaries, movies, TV, comics, coding bootcamp ads, and more) over Taylor Swift's song "Blank Space". My launch blog post on Dreamwidth goes into more detail and includes links to download it. You can stream it at Critical Commons (choose View High Quality for best experience) and I embed the video below:
It's CC BY-SA; please feel free to redistribute, link, remix, and so on, as long as you attribute me as the vidder and distribute your changes under the same license. Comments are welcome, though moderated.
# 22 May 2015, 10:48AM: Missing Women in FLOSS Philosophy, and Borrowing Models from Fandom:
I've arrived in Madison for WisCon! And just in time for WisCon:
I have a blog post up (in two parts) focusing on the frameworks that we free software/open source folks often take for granted, what might have been erased from our FLOSS intellectual heritage due to sexism, what FLOSS might look like under a different approach, and what practices and perspectives we might borrow from the fan fiction/fanvidding realm of speculative fiction and media fandom.
Part 1 is up at Crooked Timber as the guest post "Where are the women in the history of open source?" Part 2 is up at Geek Feminism as "What if free and open source software were more like fandom?"
Please feel free to comment at CT or GF.
# 22 Apr 2015, 12:14PM: How Knowledge Workers Can Learn More About Open Source Tools They Use:
Yesterday I spent an hour teaching a woman whose nonprofit wants improvements to their current Drupal setup, especially around content approval workflow and localization. She wanted to understand more about how Drupal works so that she can understand the potential problems and solutions better, and be a better partner to her technical colleagues.
I talked with her a little about those specific questions, but most of what I taught her would be appropriate to any knowledge worker who wants to learn more about an open source web application. I pointed her to some resources and figured they were worth mentioning here as well.
- The Felder-Silverman engineering learning styles questionnaire. You knew I would do this. I am such a pusher. Whenever I hear someone talk about the frustrations they've had in learning how to bend software to their will, especially if they get self-blamey or overwhelmed with approaches and resources, I suggest they take this quiz. It's helped me and other people reduce self-blame and get more strategic.
- The English Wikipedia page about Drupal. Sometimes open source projects' websites are not, to use the church jargon, "seeker-sensitive." In those cases, Wikipedia often has good summaries to answer questions like "What's the latest stable version?" and "What are key terms I need to understand to look up more help?"
- The Freenode webchat service, so you can join an Internet Relay Chat channel without having to install new software. Most open source projects have live chat channels, where you can ask questions, on the Freenode IRC network. You can make up a nickname -- it's not permanent -- and join, for instance, the channel drupal-support (guide to using IRC politely). Thanks to eevensen and ciss in that channel yesterday for tips:
[15:31] nyplguest: I'm starting to get into using Drupal - what's the best intro glossary/document to help me understand the vocab, like blocks and views? (I'm used to another system)
[15:34] eevensen: @nyplguest I recommend
[15:35] ciss: nyplguest: https://www.drupal.org/glossary
[15:36] nyplguest: Thank you ciss!
[15:37] nyplguest: Thank you eevensen as well!
- The NYC Drupal group, which in the past has run a Drupal Ladder series of events to teach and train new contributors. (I know of Drupal Ladder mostly because my pal Fureigh led Drupal Ladder in NYC and gave an Open Source Bridge talk about it.)
- The new Wikimedia content translation tool that makes it easier for you to translate articles. Maybe your website can do something similar.
- The "workflow" Drupal group, which looks like a place you can ask how to set up the workflow and content approval process you want.
- Some things I learned about domain names and hosting, and things I learned about Drupal. This included discussion of:
- "The Five Stages of Hosting" (e.g., dorm room versus condo). Such a useful analogy.
- DigitalOcean, the "dorm room"-type provider I use. It's been a good deal for what I've needed, namely, a test server that I can blow away at the slightest provocation. https://www.digitalocean.com/?refcode=82e7b02dea11 is a referral link to get a $10 credit at signup (that's 2 months' worth of service at the $5/month plan).
Since she may end up with a test server so she can play with Drupal modules and configuration, I also talked with her a bit about what it means to ssh into a server, the fact that she will probably have to install new software (a console or terminal application) on her Windows computer in order to do that, and the basics of how public key infrastructure and SSH keypairs work, and why they're more secure than just using a username and password. I did this without notes or links, so I don't have any to offer here; perhaps you have a favorite explanation you'll share in the comments?
Overall in these kinds of conversations I refrain from saying "do this" or "do that", but I did share these two bits of wisdom:
- When you generate a keypair, the .pub file is the one to give other people, and the other one you keep to yourself.
- Make an effort to remember that passphrase. Otherwise you will be unable to use your key, and you have to have a slightly embarrassing conversation where you say "here's the new .pub because I forgot my passphrase for the old one," and it delays whatever you were going to do. But I showed her my ~/.ssh directory with all those old keys I can no longer access, and told her that if she does end up needing to make a new keypair, she is in good company, and basically everyone with an SSH key has gone through this at least once.
We talked about getting her a community of practice so she could have more people to learn from. She now knows of the local Drupal group and of some get-togethers of technologists in her professional community. And she has some starting points so she can ask more productive questions of the technologists within her org.
And this stuff is frustrating, and if you feel that way, that's okay; lots of other people feel that way too, and maybe it just means you need to try a new approach.
# (1) 18 Apr 2015, 08:54PM: La Con De Python:
I spent a good chunk of this month at PyCon in Montréal, watching talks, seeing people I rarely get to see, and working on Mailman. My stay in Montréal felt homey thanks to Jo Walton and Emmet O'Brien, who put me up in their place for the duration. Much thanks, Emmet and Jo!
It was wonderful getting to sprint with the rest of the Mailman team, some of whom I'd never met before. I'm grateful to the Python Software Foundation and the PyCon sponsors for arranging the venue and food; one can attend the sprints at no registration cost, and I thoroughly appreciate that. I wrote a few patches, told other attendees about the upcoming release and got them to come test the install, and did a great deal of testing and bug-reporting myself, and generally a bunch of release management. I had the privilege of discovering a funny bug, although I wish the bug didn't exist since it prevented us from meeting our goal and shipping 3.0 by Thursday. (A 3.0rc1 release is imminent!)
On the last day of the sprints, I started a keysigning. I think every keysigning I've ever participated in has included philosophical and engineering questions about the usefulness of keysigning parties, why we bootstrap an anarchistic web of trust using government-issued documentation to authenticate people, the difference between "I control this key" and "I am the person whose passport this is," and the anti-mnemonic powers of
gpg command-line flags. I feel as though there ought to be, and perhaps is, a haggadah for this ritual that incorporates these questions. I can't exactly remember this exemplary exchange from Thursday, but it went something like:
Me: I wonder what I would learn if I tried setting up my own keyserver.
Debian guy: You would learn that the system is utterly ripe for abuse and that we're just lucky no one has seriously tried it yet. It's an append-only distributed database, after all.
Me: (Pause.) I think I had already learned that particular social lesson and I was thinking more of the technical lessons.
Debian guy: Ah! Yes, there are some interesting backend protocols involved....
This was the longest stretch I've ever spent someplace Francophone, and I felt my high school French coming back to me day by day; towards the end I was able to put together "J'ai perdu un chapeau bleu" or "Je voudrais acheter cette chose" with tolerable facility. (I did indeed lose a blue hat that I bought in Washington, DC in 2001 just before I left for my trip to Russia; we had a good run together and I hope it ends up with someone else who likes it.) I have never played Flappy Bird, but I understand that a single error ends the round; similarly, bad French in Montréal is a sudden death game for me, in which a single mistake or even a tilted head while parsing a response can cause the interlocutor to switch to English. Like many people with one dominant language fluency and a lot of language smatterings, I find the wrong language's vocabulary springs to mind at inopportune moments. A caterer was serving me food; I couldn't remember the polite French for "that's enough" and my mouth wanted to say "ಸಾಕು" instead. Similarly, "mais" and "et" no longer come as naturally to me as do "но" and "и". But I have it easy -- evidently this is even less convenient when one of the languages is ASL!
The next PyCon North America will be May 28 - June 5 2016 in Portland, Oregon; this overlaps with the Memorial Day weekend in the US (May 28-30) which means it will probably conflict with WisCon's 40th anniversary, and I already have plans to hit WisCon 40. I hope to finagle schedules so as to attend WisCon in Madison and then fly to Portland to participate in post-PyCon sprints. But that might be too much spring travel, because what if Leonard and I want to do something special in April to celebrate our tenth wedding anniversary? What I am saying is that adulthood sure does have a lot of logistics involving calendars.
# 18 Apr 2015, 12:36PM: New Takes On My Published Writing:
My Crooked Timber guest post on codes of conduct, freedom, governance, contracts, and copyleft software licenses has attracted over 200 comments. Many of them are thoughtful and interesting, and worth at least a skim if you found anything useful in the original post. For instance, can we compare mindshare to other forms of property? And what do we do to legitimately obtain the enthusiastic consent of the governed? Some of them have old or new perspectives on Adria Richards or Linus Torvalds. And about five percent of the comments are gross, hurtful, or laugh-out-loud wrong on multiple axes, e.g., "The FOSS world is not asking for codes of conduct, she is seeking to thrust them upon it." I shall be mining those for use in my stand-up comedy routine at AlterConf in Portland, Oregon in June.
Also, the code4lib Journal asked for me to turn my code4lib keynote from 2014 into an essay, "User Experience is a Social Justice Issue", for their special issue on diversity in library technology. The new article includes some contextual introduction and a retrospective with links to related work by others in the past year. You can comment there.
# 10 Apr 2015, 11:10PM: Crooked Timber Guest Post on FLOSS Licenses and Codes of Conduct:
The social sciences group blog Crooked Timber has published my guest post, "Codes of conduct and the trade-offs of copyleft".
A lot of open stuff -- such as the Wikimedia/Wikipedia and Linux projects -- are discussing or adopting codes of conduct, or expanding their existing policies. I'll reveal my biases at the start and say I think this is a good thing; for more, read my speech "Hospitality, Jerks, and What I Learned". But in this piece, I want to talk about the similarities and differences between codes of conduct and a set of agreements that some of these communities are more used to: "copyleft" or other restrictive software licenses. And I'd like to draw out some ways that the kinds of acts and artifacts that these policies cover reveal different attitudes towards contracts and governance.
Also I make silly references to Antitrust and Ducktales while oversimplifying free software licenses and political theory. So check it out.
Much thanks to Skud for an initial conversation about face-to-face versus online codes of conduct; my article, in the end, barely addresses that, but it was a seed for this piece. Thanks to Henry Farrell of CT for editing and publishing my guest post. And thanks to Naomi Ceder, Paul Tagliamonte, Leonard Richardson, and several other people who talked about this topic with me or beta read bits or drafts of the piece -- of course, all errors are mine.
Feel free to comment over at Crooked Timber!
# 13 Jan 2015, 03:39PM: Unlocking The Funhouse (Mirror):
In technology (as in many communities), capitalism makes it hard for us to understand what we're good at. A few source texts, and then a sketch of some contours.
- The "No true Scotsman" fallacy.
- Shweta Narayan on category structure, cognition, and side effects.
We tend to have this idea that categories, like "bird" or "food" (or like "human" or "white", which is what this is all really about) are like solid boxes. Entities are either in them or out of them, with a clear and unchanging boundary, and everything inside is an unsorted & equal jumble, and everything outside ditto.
This notion gets strongly underscored by our cultures, so it can be hard to ... er... unpack. But the fact is, cognitive categories aren't actually like boxes. They have internal structure, and fuzzy boundaries (which people can draw in different places, and move depending on context), and these things matter hugely in how we think about and deal with oppression....
we need to be aware of category-centrality as well as membership....
- Huckleberry Finn, specifically:
"All right, then, I'll GO to hell"--and tore it up.
The nuance I still ponder is: Huck doesn't say his way is right. He decides he's wrong but he's going to do it anyway. He decides to be a hypocrite. He does not see himself as articulating a new consistent ethical framework under which he is morally right; he is accepting the status and the consequences of his actions in the religious framework everyone's taught him, but he decides not to let that get in the way of what he feels compelled to do. It's a different kind of resistance.
I heard an echo of this moment in "The Rundown Job" (Leverage, S05E09), when a government official tries to get Eliot, who used to do wetwork, to leave the Robin Hood-type vigilante outfit he's with now:
Colonel Vance: The world can always use more good guys.
"Why Job Titles Matter To Me", a piece I wrote last year.
Deb Chachra on discomfort with the identity "maker" and the primacy of "making".
Eliot: Yeah, well, too bad we're the bad guys.
I educate. I analyse. I characterize. I critique. Almost everything I do these days is about communicating with others. To characterize what I do as 'making' is either to mistake the methods -- the editorials, the workshops, the courses, even the materials science zine I made -- for the purpose. Or, worse, to describe what I do as 'making' other people, diminishing their own agency and role in sensemaking, as if their learning is something I impose on them.
"MDN MozFest outcomes: self-teaching", a summary by Jeremie Patonnier that said one of the tools that self-directed learners most want is "Tools to measure/evaluate one's level of knowledge."
You may not be able to tell from this blog that I, like many people in tech, do experience self-assessment vertigo. Software engineering includes a zillion skills (it's clearly not just computer science) and no one knows all of them. We're so bad at assessing who's good at what that we end up pronouncing that the only way to tell whether someone is "good" is to work with them, or we use "culture fit", personal recommendations, and other easier-to-grasp handles as lossy proxies. The bizarre informational distortion of the job market makes it even harder to get a clear picture of one's own skills, "objectively" and relative to others. Even if, like me, you are not currently looking for a job as a programmer!
Outside of academia and Hacker School, the primary way I hear people talk about technical skill assessment is in relation to the job market or job titles. (And even in academia it's early days yet in teaching software engineering.) In open source we sometimes make one-time assessments as to whether individual people are ready to become maintainers, but other than that, the discourse I hear is about matching candidates with paid employment, and so we assess ourselves and each other in terms of potential job titles.
Just as there is no inherent genre to books (the "genre" of a book is a way to market it to the readers who would like it) there is no inherent category "backend engineer" or "business analyst" etc. That's just a convenient name that we have socially constructed to kind of correspond to a set of skills. (And so the goalposts move so easily it's as though they're on casters freshly sprayed with WD-40 by someone shouting "But no true hacker...")
Within individual organizations, there's some consistency in what a particular job title means. But the job descriptions the public sees are often wishlists that don't distinguish between "desired" and "required" qualifications for a particular title. And a "hey you're interesting for position x" email from a recruiter gives us a data point, even if it's super wrong, and maybe even so wrong that it is demoralizing to candidates! ("Shit, the only recruiters who reach out to me are so dumb and desperate that they don't count" or "Crap, I still look like a foo instead of a bar".) We get a lot of noise mixed in with the data.
My particular set of skills does not correspond to any particular well-known bucket, and I should not let that make me feel bad.
Buuut of course socially constructed things are real too! And it is useful to know whether I am correctly performing the role of "fullstack developer" or "devops expert" or "community manager", to know whether I can attract the particular kind of attention I want! And it's useful to know when I should say, "yes, according to the tech industry's dominant hierarchy, the work I enjoy and think is most important marks me as low-status, unintelligent, and ignorable. So what."
Even if I can get away from looking at myself as a good little worker bee, impostor syndrome and Dunning-Kruger both affect self-assessment. While I believe I am fighting both, it may be unavoidable that the only way to get better at self-assessing a skill is to get better at the skill in question, reflecting all along the way. Thus: a code review group. (Check out how I briefly describe my programming skill level in that post, by referring to what I can and can't do.) Thus: my Mailman work. Thus: blogging. Sketching out where I am so I can see where I've been. These points of data make a beautiful line.
Edited on 6 Feb 2018 to add: I said some of this stuff better in my post today, The Ambition Taboo As Dark Matter.
# (1) 25 Dec 2014, 12:48AM: Good And Bad Signs For Community Change, And Some Leadership Styles:
So let's assume you want to improve a particular community, and you've already read my earlier pieces, which I am now declaring prerequisites: "Why You Have To Fix Governance To Improve Hospitality", "Hospitality, Jerks, and What I Learned", and "Learn Tech Management in 45 Minutes" (all the way through the Q&A). And let's assume that you care about the community having a good pathway to inclusion, and that the community is caring or collaborative, rather than cordial, competitive, or combative.
When I look at an open stuff community, here are some factors that
make me optimistic:
- people with social capital in the project, whom other participants respect, support my goals in private conversation
- even better: such people have reached out to me, of their own
initiative, about it
- even better than that: such people are already taking real action
- I have personal relationships with at least one influential project leader
- I am in the private spaces where project leaders talk
- either the project's still new and the norms are in flux, or there's a new initiative or subcommunity where I can influence norms or even amend the rules of the game before they jell and harden
- the founder of the project exercises charismatic/inertial authority and either does not support my goals, or is too afraid of conflict to take real action
- per Selena Deckelmann's advice, "If someone is treating you with contempt, or you are using contempt in arguments, that's a big warning sign."
- there is a private space where important conversation happens and I'm not invited
- I, or someone else who shares my goals, has been unsuccessful in
getting the community to do something small towards my goals. For instance, assuming my goal is improving gender diversity in a male-dominated workplace, I haven't been able to get them to adopt a first code of conduct, or improve a CoC to have real enforcement provisions, or participate in a women-centric job fair, or make a token effort towards diversity in guest speakers.
- not just the rules of the game, but the dominant worldview, and perhaps the major actors, haven't changed in, say, more than three years
To achieve change in this kind of situation, you have to have enough social skills to be able to make relationships, to notice whether contempt has made an appearance, to grok the subtle stuff. A systems approach (leader as engineer) will get you part of the analysis and part of the solution; you also need relatedness (leader as mother). Requisite variety. In the face of a problem, some people reflexively reach more for "make a process that scales" and some for "have a conversation with ____"; perhaps this is the defining difference between introverts and extroverts, or maybe between geeks and nongeeks, in the workplace.* We need both, of course - scale and empathy.
A huge part of my job for the last four years was struggling with the question: how do you inculcate empathy in others, at scale, remotely? How do you you balance genuine openness to new people, including people who think very differently from you, with the need for norms and governance and, at times, exclusion?
Huh, I wonder whether this is the first blog entry I've ever tagged both with "Management and Leadership" and "Religion".
# 03 Jun 2014, 08:39AM: Choosing Older Or Younger Open Source Projects To Work On:
Larger, older open source projects have more people, more getting-started resources for new contributors, more name recognition, and sometimes more money to spend. (Examples: the Linux kernel, MediaWiki (the software behind Wikipedia, part of Wikimedia), Mozilla (the makers of Firefox), WordPress.)
Younger ones, with smaller contributor populations and smaller codebases, sometimes give new contributors more responsibility and power quickly, change faster in response to new ideas, and have more malleable culture -- and you can become one of the few World Experts in that technology more easily. (Examples: Tornado, ClojureScript, MetricsGrimoire, ThinkUp.)
So, while Mozilla, GNOME, Wikimedia, etc. have bigger budgets and more formal programs, and often have a larger worldwide impact, it could be that smaller and younger projects will give you more relative expertise faster. It's worth considering.
(You can use Ohloh to find open source projects on a particular topic, and see how many contributors they already have, and to compare projects. Take the statistics with a grain of salt, though; sometimes they're off.)
# (2) 26 Feb 2014, 07:10PM: Some Help for New Open Source People:
Wikimedia is participating in this year's Google Summer of Code internships and Outreach Program for Women. This week we are seeing a bunch of new folks try to learn how to navigate the world of open source, and I have some advice for you. Some of this ought to go into the Google Summer of Code student manual and the Open Advice collection.
"Doubt": Lots of GSoC candidates are from South Asia. Indians often say "Can you help resolve my doubts?" where US speakers would say "Can you help answer my questions?" "Doubt" and "question" are synonyms here; the Indians aren't implying suspicion.
How we talk: We talk in different places when we want to have different kinds of conversations. Each open source community has "a mailing list, a wiki, and an IRC channel.... a platform for discussion, storage for documentation and real-time communication." (I borrowed this explanation from the hackerspaces wiki.) An IRC channel is a constant waterfall of conversation and you aren't expected to be there all the time or catch everything. A mailing list is more like a slow-moving river, and a wiki changes slower, like a marsh.
Some people prefer for their IRC conversations to be more like mailing lists -- a long, publicly archived conversation where people can see what happened before and take part. Some people prefer for IRC chat to be more like Snapchat -- ephemeral, temporary, so it's easier to be vulnerable. No one agrees on what all of IRC should be. So the community within each channel has a certain culture and each channel can be different. Some channels allow or encourage public logging (example) so anyone can see what happened in the channel. Others don't. This difference is normal.
The rhythm of help: When you are learning how to contribute in open source, you're going to find that people give you links to pages that answer your questions. Here's how that usually goes:
This helps us make a balance between person-to-person discussion and documentation that everyone can read, so we save time answering common questions but also get everyone the personal help they need.
- you ask a question
- someone directs you to a document
- you go read that document, try to use it to answer your question
- you find you are confused about a new thing
- you ask another question
- now that you have shown that you have the ability to read, think, and learn new things, someone has a longer talk with you to answer your new specific question
- you and the other person collaborate to improve the document that you read in step 3 :-)
What's this project like?: Figuring out whether something's a good project for you is a skill and new folks don't have that skill yet. My friend Mel wrote a guide to how she checks out an open source project -- how she takes five minutes to look on their website for certain things, to see what kind of project it is. It's fine for you to look for projects where you already have friends, or where they have already set up easy tasks for beginners. We hope that in a year you'll be one of the people coming up with new ideas, organizing those easy tasks, and helping the beginners.
# 06 Oct 2013, 12:00PM: What They Don't Know:
Or: you are an expert if you can save people time.
Late in 2011, I found out that one of my colleagues, a whip-smart and infinitely organized administrator, wanted to know more about how the engineering side of Wikimedia works. So I started teaching her. Every month, we talked for about an hour. She asked me about some activity from the monthly report and I explained what we're doing and why, often using analogies. She loved it and felt far more connected to what her other colleagues were doing.
She's not at Wikimedia anymore, so I have tried doing it as a Wikimania presentation and continuing the tradition with other WMFers who were interested. So far I've done a lot of one-off "What the fudge does Wikimedia engineering do" sessions for incoming folks, mostly non-engineers coming into the Foundation's other departments.
Two lessons from that experience:
- Sure, continuing mentorship relationships are awesome. But don't discount the value of a few limited teaching sessions.
- I have about three approaches to teaching this stuff: Historical (What has happened since we started in 2001?), Experiential (What happens under the hood when you go to en.wikipedia.org in your browser, and who's in charge of what parts?), and Organizational (Who are the eight directorates in WMF engineering, and who are other important Wikimedia tech institutions, and who does what?). I want to get better at the historical mode, which means learning what happened in what order between 2001 and 2011; right now I do the org-chart mode quite well, and the experiential mode well except for talking about load-balancing and caching.
I wish I'd kept good notes of all the questions people have asked during these sessions. Some of them:
- What is a parser?
- What is LAMP?
- What is MySQL? What is a database?
- What is Apache?
- What does "open source" really mean?
- How can it be that so many talented programmers are only in their twenties?
- What is the role of the Engineering Community Team?
- What do the people in the MediaWiki core team do?
- What is Subversion, what is Git, and why did we switch?
- Do all the Wikimedia sites run on MediaWiki?
- How can we do what we do with so little staff?
- What's with this Lua thing?
- Why has it taken so long to write the Visual Editor? (This question led me to sketch out a blog post we published.)
- What is a "virtualized hosted development environment" (Labs)?
- Why did we have to switch to IPv6 and why was that hard?
- What is an API?
- What is HipHop and why would we use it?
- Why did we work on a specialized Wiki Loves Monuments app?
- What are the Universal Language Selector and Milkshake?
- What is Swift?
- What is the difference between the E2 (Editor Engagement) and E3 (Editor Engagement Experiments) teams, and what do they do? (We partially fixed this by rearranging and renaming the teams to Core and Growth.)
- What is HTTPS? How does SSL work?
- What kind of security problems could a web-based application have? Do we have worse problems because we're open source?
- What does a product manager do?
- Why don't we provide automatic translation from and to different language Wikipedias?
- What is Wikidata?
- Is Wikipedia Zero just for Wikipedia, or also the sibling sites?
- How do we consult with hundreds of different wiki communities when building and rolling out our software, especially when we don't speak their language?
I have just started at Hacker School, a place designed to help everyone learn. That means making people feel comfortable with saying "I don't know". I've benefited countless times from this, because if no one's going to belittle me for not knowing something, I feel safer asking and learning. I didn't realize how much I would also get to teach! When everyone feels safe saying "What does that mean?" then I get to help more people learn more things. I've explained, among other things:
It's super amazing when you teach someone a skill or a perspective that changes them. I feel so lucky that I am an expert, i.e., someone who can save other people time. It is a form of hospitality.
- what Markdown is, and why you would use it
- what screen-scraping is, and why APIs would be better
- how I use git
- what unit tests are, and why you would add automated testing to your project
- dozens of opportunities to reuse, integrate with, or improve Wikimedia data and software
- a bunch of Unix command-line tips, such as control-R for interactive search of bash history
- the sordid history of ReiserFS
- why Nvidia drivers are the classic example of "proprietary stuff that's not in the Linux kernel but that you might want to use so some distributions carry it"
# (1) 12 May 2013, 09:49AM: Tips for New Summer Interns:
Three tips to help new Google Summer of Code applicants and interns, some of which all remote workers could stand to remember:
- Never let yourself get stuck on a technical question or problem for more than half an hour. Take a break, ask questions in IRC or a mailing list, find a technical book to read like
The Architecture of Open Source Applications, look at some other codebase to
see how they do it, eat a meal, or do something else, then come back to
- Never let yourself get stuck waiting for someone's reply for more
than 2 business days (Monday through Friday). Escalate -- ask your
mentor. If your mentor isn't helping, ask your org admin. If the org
admin isn't helping, ask on the GSoC discussion forum, or email Carol Smith.
- Ask yourself at the start of every day: what did I accomplish
yesterday? What will I try to do today? What are the obstacles I think I will run into? If you ask yourself those three questions and answer honestly -- especially if you let your mentor and team know the answers -- then you will prevent long delays and help keep your morale up.
# 16 Feb 2013, 07:01PM: Navel-gazing:
There are so many things I ought to be doing, and instead I spent several hours today editing Wikipedia and Wikivoyage, reviewing new articles, and uploading photos or improving captions on Wikimedia Commons.
It makes me think of that panel I was on, a million years ago, about guilty pleasures, back when I had the spare energy to go to non-work conferences. One thing I wish I'd thought of to say then: If it weren't possible to run away from "obligations" then they wouldn't be obligations, the kinds of responsibilities we encourage with norms and shaming and praise. They'd be facts like mitosis. The discourse around guilty pleasures is part of how we manage the pressure to fulfill our responsibilities to each other, a loophole that helps us avoid talking about unfair burdens.
You've heard that frontier thesis, that it's an important release valve to be able to go someplace no one knows you so you can reinvent yourself, the idea that right now is a significant historical aberration because your old identity will follow you wherever you go unless you engage in a coverup, that the defaults have flipped. The productivity frontier is somewhere in this danger zone as well, and I can see the temptation to Taylorize myself and those around me, and perhaps my workaholic ethic is so strong that even my guilty pleasure is reducing New Pages Feed backlog.
# (2) 22 Mar 2011, 08:45PM: A Slightly Disjointed (Due To A Five-Day Cold) Musing On Open Source, Fear, Motivation, And Witnessing:
I was introducing C. to a set of QuestionCopyright friends and acquaintances, and they were joking about indoctrinating her, and she was curious to hear what free culture is all about. So she wondered why I reflexively suggested that the others wait a bit, tell her next time.
They did give C. the introductory spiel, and conversation was pleasant and edifying, and nothing terribly awkward ensued. She has developed a substantial interest of her own, now, in the theory and practice of free culture. But why did I have that reflex? I felt around for it and grasped something. It makes it harder, I said, once you know these things and care about them. Becoming a free culture/free software person is like becoming a vegan.
No, G. replied -- at least people know what vegans are.
We happy few.
Here I was, a fulltime free culture/free software consultant, feeling an unaccustomed reluctance to give someone else the sunglasses, to witness.
There are self-constraining ideologies like veganity or chastity that modern society at least theoretically understands, even if some cohorts scoff. Then there are the practices that always require an introduction. When I explain how I met Leonard, I often start with the thirty-second "what is open source" explanation, because it's all of a piece. But my "what is open source" intro focuses on pragmatism -- many eyes making bugs shallow -- rather than free software values.
I think I'm a moderate sort of open source gal, an ovo-lacto vegetarian. There's an iBook running Mac OS tucked off in a drawer, and all these Linux boxen in our house surely have nonfree binaries driving bits of hardware. No Facebook but I surely use many cloud services that violate the Franklin Street Statement. I hang out with copyright abolitionists, Debian users, and other free culture/free software folks who make me feel namby-pamby. And then I go to dinner with someone who makes me feel like a Jain. Or I find myself saying, as I said a week ago, that developing on a closed platform is like trying to fall in love with someone who won't talk to you.
Our love is part of what energizes us, moves us to act. In FLOSS, volunteers do things for two basic reasons: either because we enjoy doing them for their own sake, or because the task needs doing and we want to do our bit. We see some goal the task will help us reach, or fear an outcome the task will help us prevent. [By the way, it's useful to have experienced that, because it's useful to assume those two as the means of persuasion whether my colleague's paid or not. As a leader, I should either set up tasks people will genuinely enjoy (and get the scutwork out of the way), or help my colleagues see a straight line from the task to a glorious future. Show them how what we're doing leads to something they want. This is my pet theory of How To Lead Knowledge Workers and your mileage may vary.] And -- as a zillion social scientists will tell you -- even if we momentarily burn out on caring about a goal for its own sake, we don't want to let the team down. We don't want to let our buddies down.
As we were talking about GNOME marketing, Andreas once asked me what I found special, what personally spoke to me about GNOME. I rambled: object code is compiled from source code, but the source code is compiled, too -- compiled from people, from time, from love. Every time I look at my desktop, every feature and every bug comes from someone, someone with a name and a face, and sometimes I can even remember. Hey, I remember when she added that feature to Empathy. Oh, right, I know he's working on that bug. It's like all of Planet GNOME is helping me out, every day. It's like my whole community's right there, on my desktop, every time I open the laptop lid.
I don't want to keep my friends blissfully ignorant of this. Is there a more loving human impulse than the joy of sharing? I'm sorry, C. I'm sorry I was afraid of making your life harder. I remembered the local minimum and forgot the greater maxima awaiting you. Why keep us a "happy few" when we can be an ecstatic many? And yes, it's harder, to learn our principles and try to walk this path alone -- but the whole point of our principles is that our multitude, our diversity, our union, our communion is far richer and more sustaining than individual hoarding ever could be.
# 15 Mar 2011, 01:24PM: New Edition Of/Nueva Edición de GNOME Journal:
In this special edition of GNOME Journal, GNOME HISPANO's Juanjo Marin arranged for us to get five great stories in both English and Spanish. You can read these now at http://gnomejournal.org:
The GNOME Journal features original content and commentary for and by the GNOME community. All articles are published under the Creative Commons Attribution-ShareAlike 3.0 license. Please feel free to translate, podcast, repost, etc.
Thanks to Juanjo, the authors, Diana Katherine Horqque, Will Kahn-Greene, Sriram Ramkrishna, and Paul Cutler for their work on this issue!
Issue 23 will come out around April 3 and focus on the release of GNOME 3.
# 27 Jul 2009, 09:42PM: Those Annoying Isms Keep Isming:
I was at a software conference a few weeks ago where Richard M. Stallman said something unfortunate. I wasn't there for it so I heard about it afterwards and shook my head and sighed.
At the conference I got to meet lots of cool developers, such as Matthew Garrett. Matthew hacks, tells fun stories, and enjoys inflicting Hackers and other like movies on his friends. Matthew also does ally work. Right now he's painstakingly explaining to the less clueful members of the free software community why Stallman's remarks were inappropriate, and why it's right and proper to criticize him publicly.
I also met a male developer who asked me what the deal was with my buzzcut hair -- was I a lesbian? If not, why the short hair? I asked him back why he had short hair, and why he didn't wear heels and lipstick, etc. Probably could have made a better comeback there.
So don't worry, I got my quota of feeling othered, despite missing that keynote. And I'm guessing no matter where I go or when I join a tech community I won't be lacking for my recommended daily allowance of genderfail.
Some of y'all don't know why Richard M. Stallman is simultaneously important and unimportant to software people like me. He did some really important stuff a few decades ago, he has a tremendous legacy, and he's ended up as one of the high-profile faces our community presents to the outside world. But these days he's talking more than doing, and he acts really touchy, and just over and over ends up saying things that make everyone wince.
At this point, if you are a science fiction fan, you might light up and say, Oh, I get it, he's like the Harlan Ellison of open source!
And WHAT DO YOU KNOW! While I was visiting my gentle geeky friends in Boston, Ellison posted some really jerky paragraphs about an acquaintance of mine. He later apologized in his own not-very-apologetic way. The whole incident made me look back at the one personal interaction I've had with Ellison (summary) in a less-than-flattering light.
I knew, years ago, that I'd have to deal with crap from the communities that I loved, because of my heritage and my chromosomes. But I didn't know, viscerally, how tiring it would be. The more I accept my membership in these communities as a part of my identity, the more headspace these incidents take up. If I work hard enough, contribute enough, maybe someday no one will dare say I'm not good enough. Maybe someday I'll reach tremendous stature in my chosen community, and turn into the token nonwhite/female who gets used as proof that We Aren't Bigots, Really. A depressing thought.
I'd like a future where my race and sex are never the most remarkable things about me, in my work or in my hobbies. It's International Blog Against Racism Week. Because I'm not the only one who thinks that way, thank goodness.
 I've tried emacs a bunch of times, including periods of sustained use, and I know I need to actually put the time and energy in to grok it, learn all the keyboard shortcuts, at some point. Neal Stephenson on emacs increases my desire to do this; RMS/Yegge do not.
# (3) 24 Feb 2009, 12:39PM: Two 101-Word Stories Inspired By Fred von Lohmann's Talk Last Night:
Most clients sounded more stressed and less grammatical than this guy. "Why did YouTube take down a video without soundtrack music? I didn't break any copyrights, did I?"
"You came to the right effing lawyer," O'Porter smirked, though technically EFF had fired him when he kept calling Seth a "Latin hunk." "Let's see it."
The stranger clicked Play and swiveled his laptop. O'Porter watched hamsters and tried to hear the words under the strange hiss --
Seth David Schoen closed the lid, peeled off his mask, and walked away from O'Porter's body. Really, breaking the Content-ID tool was just a bonus.
"I'm saying, 'Leibnitzian Python wonder-language that contains no ambiguity' was a JOKE, not a spec."
"So he was a jester-philosopher, the Birbal of his day."
"I think Colbert, Haskins or Stewart --"
"If code is law, shouldn't law be code? And who'll port it but us?"
"But it's the Cyc problem. We write legislation using subjective moral distinctions that change over time. Barring Seldon-level sociological prediction, your version 1 architecture is going to include something as abhorrent to future Americans as slavery is to us. Worst. Legacy. Code. Ever."
"Not if CSAIL works with us," said the dean of MIT Law.
Also inspired of course by Leonard and by Brendan. Very much not inspired by anything Seth or anyone at the Electronic Frontier Foundation has ever done.
You can hire me through Changeset Consulting.
This work by Sumana Harihareswara is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.
Permissions beyond the scope of this license may be available by emailing the author at email@example.com.