Categories: sumana | Programming
Software development, system administration, and similar
# 07 Apr 2021, 01:07PM: Painstakingly Reminding Myself How To Play:
This is a little bit about how free-range learners in programming assess our own skill levels and choose what to learn next. But it's also a response to my own insecurity, and to the sometimes-stultifying weight of concentrating one's work on infrastructure.
Working on things that matter
In the Abstruse Goose comic "Computer Programming 101", a learner provokes an explainer with further and further questions about the CS and hardware and physics underlying a programming task. One reading of the comic: "Get comfortable with abstraction. If you try to understand how everything works, you'll get nothing done."
Yeah, of course, everyone's time is finite, and we all have to make our own decisions about how much time to spend on learning and how much time to spend doing other things, using our existing levels of knowledge. (Although I've recently tripped up on the assumption that the listener aims to get anything in particular "done".)
But there's also a kind of obliviousness that is so helpful, not just cognitively but emotionally, when I'm learning. Not knowing that something is risky, or not really being able to comprehend risk, helps you do it. This is one reason it can be useful to learn a bunch of programming skills when you're young, not just because very little responsibility rests on your code's shoulders, but also because at that stage you haven't yet seen all the vulnerabilities and Daily WTFs and unlocalized sadnesses... you don't even know what all edge cases exist in the world. You can take the leap of faith that all your infrastructure will work -- heck, you don't even know what infrastructure you're relying on! You don't even realize you are taking that leap of faith! -- and concentrate on getting your corner just right.
For context: for my job, I primarily work, and want to work, on mature open source software that many users already depend on. I find a lot of satisfaction in rejuvenating and stabilizing widely-used open source projects and thus healing important parts of the whole system. My professional experience is loaded up with working on stuff like GNOME utilities and MediaWiki and the Python packaging toolchain. (I left the Wikimedia Foundation partly to mess around with blank slates and without legacy infrastructure/stakeholders.... and then turned into the de facto community manager for Python packaging!) I played with BASIC as a child, and I learned a bit of Scheme and bash in college, but I came to programming in a serious, sustained way AFTER years in the industry, as a technologist and manager in software engineering teams.
Which means that when I do want to make a little toy, sometimes it's been hard for me to just come to it with learner's mind. I see that it has no unit tests, no localization, a bad UI, crappy OO, no extensibility and zero separation of concerns, ridiculous performance. There are at least five worlds of software development (that article is pretty obsolete but its point is reasonable) and I am a permanent resident of the People You Don't Know Will Need To Use This Software world. I spent some of my childhood in Playing Around world but it can be hard for me to remember how to get around there. (And oppressed people, out of necessity, often mitigate risk more, tempering audacity. So that's yet another privilege thing.)
As Amandine Lee writes: "People's intuitions and risk-friendliness also vary based on personality, and how they've seen things fail in the past." Yes! But then the very next sentence: "A lot of growing as an engineer is fine-tuning that initial response to design decisions." She meta-cautions us against knee-jerk caution, a reflex that leads to "wasteful carefulness". Was it nearly a year ago I talked about this, about the balance between preservation and growth? Maybe it's a springtime kind of rumination.
Precursors to relaxation
I am trying to think about what helps me let go of those worries and fiddle, sketch, prototype. Curiosity about a specific dataset helps, as does the impatient desire to munge some data into a form I can more easily reuse. Or an external force causing me to concentrate on achieving some specific outcome OTHER than "other people need this," like "I want to create enough of a game that I can put it in my application to the Recurse Center" (e.g., this commit in "Where on the Oregon Trail is Carmen Sandiego?" -- global variables and pretty naive string concatenation abound, as you can see, and I think those were the first two classes I ever wrote). I also think it helps when I feel like I am exploring abundant neat stuff left over by past architects, as with "HTTP Can Do That?!" (video).
Geoffrey Litt reports that part of it, for him, is concentrated time: "Also, I just gotta say: years of professional software engineering has trained me to work sustainably, but there's something to be said for a few long, unsustainable days of furious programming. Early-stage creative prototyping seems to benefit from a certain energy level that's not easily attainable in a sustainable environment." (Which makes me think about different ways participants can use Recurse Center, deliberately creating bursty rhythms of work and recovery, if they're concentrating on inventing, versus using a consistent routine to aid learning.)
Security and insecurity (how novel, I know)
A few years ago* I started thinking about how to harness this dynamic for play and confidence, specifically by improving my cybersecurity skills. My reasoning went:
- I often see good engineering that is better than I could do
- There is a counterproductive reaction-pattern in my head that sometimes finds it intimidating, not inspiring, to see amazing work
- Thus I get turned off in a fixed-mindset way, thinking "I am not a good engineer" because of my relative inferiority
- But the reverse is also kind of true; if I discover flaws in real running production code then I will notice my relative superiority and feel more confident about my own abilities, which raises my morale and makes it easier for me to try things that I might fail at
- There is a lot of poor engineering out there, especially when, for instance, viewed through a security lens, and it is probably possible for me to use existing resources to understand common flaws and learn how to find them
- Thus, it would be a good step for me to learn more about the bit of the software industry that has lots of terribly written code, in production, that I can inspect and feel superior to
(There's something here in common with what I've said about ways to deal with impostor syndrome, and self-assessment vertigo -- find reminders of my own competence as compared to the whole human population, not just the experts whose skill level I aspire to.)
Less coherently, I feel emotionally insecure and feel digitally insecure; I would like to be able to make better-reasoned tradeoffs about my digital behavior and protection. And I was noodling around, thinking about the community of practice of script kiddies, and the envy I feel when thinking about having the time and equipment to play like that, and the joy of feeling powerful but not responsible. I thought that would be something I would get out of offensive (rather than defensive) security skills: a feeling of power without necessarily then feeling a new weight of responsibility.
Fast forward to now. I went in approximately the opposite direction. Sure, I know more about cybersecurity now, and I'm even a visiting scholar in an academic lab working on cybersecurity. But it's to better secure the Python packaging pipeline! More infrastructure work! I have not learned any offensive skills and all of my power comes with responsibility! It's like the sitcom trope where a person says "I think I'm gonna skip that party" and then the show cuts to them seated in the middle of a big banquette table at the restaurant and everyone's wearing party hats.
And I now know myself well enough to know that, as soon as I notice a needless wasteful problem, I itch to fix it, and have to remind myself to pick my battles. So: even if I did grow in my offensive skills, every time I noticed a vulnerability, I would immediately feel a frustrated desire to patch it, more than I'd feel a confirmation of my own capabilities. I am too mature to have power without feeling commensurate responsibility. I missed my window.
Old advice for a new mind
A few nights ago I couldn't get to sleep because of a wave of insecurity and negative self-talk. I never went to MIT and I wasted my social opportunities in college and that's why I founded Changeset solo instead of with a cofounder and that's why I haven't yet achieved what I wanted to! I'm middle-aged and my neuroplasticity is declining and it's too late for me to gain momentum on improving my habits and getting more efficient and making an impact! That sort of thing.
And I remembered an old teacher of mine, Mr. Berkowitz. He taught government and economics at my high school, and he looked ancient and frail -- when he slowly walked the path between the administration building and his classroom, I thought I could see the wind threatening to knock him over. And that's why it made such an impression on me when, on the last day of class, he told us: "if you keep learning, you will never grow old."
And I got out of bed and went to my computer, and figured out how to install Rust (with help from 2 people in the Recurse Center's Zulip chat), and started Rustlings, an exercise-by-exercise approach to learning Rust by fixing code that doesn't work. I completed the first exercise and got the string of "tada" emojis and smiled, a strong real spontaneous smile, and felt and noticed it. And a few exercises later, I was calm enough to go to bed and fall asleep.
I have some unformed ideas about how knowing a bit of Rust might help me with my work, to lead projects like Federico Mena-Quintero's work on librsvg, replacing C library code with Rust. But maybe the big reasons it appeals to me are that everyone I've ever heard of working on Rust is friendly, and the language aims to be really helpful with its error messages, and no one needs me to learn it. It's ok if I don't do it. Which makes it more ok to do it.
In my job I want to work on things that matter. To do that job well I need to learn. The pressure of "this matters" can make it harder to learn. Therefore there is meta-work I must do to make tidepools and sandboxes for myself to learn in, shifting my mindset accordingly. And, for bigger jaunts into Playing Around World, maybe making time for another retreat at Recurse Center sometime.
* I have a note here that maybe this was related to my experience watching a preview of Jessica McKellar's talk "Building and breaking a Python sandbox". In it, McKellar mentioned to us that ping runs as root, which stuck with me.
# (1) 22 Mar 2021, 09:09PM: A Spec For A Sandboxed Open Source Project Environment:
I'm writing a book on maintaining legacy open source projects to help teach people vital skills. Right now, as far as I know, there's no textbook or course you can work through to learn skills like assessing an open source project systematically, triaging bugs, noticing quiet-but-promising contributors to promote, improving code review processes, writing a grant proposal, and finding your successors. Or, more accurately, there are courses and guides that cover different software leadership areas, but there's nothing that covers the whole toolbox.
When I'm teaching skills, I want to give learners exercises they can use to develop and practice those skills. And if I turn the book into a course, I'll want to be able to assign those exercises and review their homework. So I started thinking: how nice it would be if I could snapshot or composite together a sample legacy FLOSS project, complete with messy old issues, docs, and chat and list archives, and replicate it in self-serve sandbox instances for exercises!
I've been thinking about this for a while. Today I wrote up a spec. I don't know what to call it - "Maintainer Sandbox", "Snowglobe Factory", and "Diorama" all sound good. The spec below assumes that I'd be leading a cohort of learners through a semisynchronous online course; it would work fine for an in-person class as well, but I'd have to adapt the access levels part for a completely self-paced and self-driven course.
As instructor, I would create a snapshot of a sample open source project (comprising materials listed below). The hosting platform would replicate it in self-serve instances, usable over the web, for user exercises. Upon signing up for a course, a user would get access to a freshly provisioned instance, complete with project history.
Materials: Each instance would include the following artifacts, all browseable and searchable via the web browser:
- Bugtracker-type artifacts and history that would generally be available as part of a GitLab or GitHub project: a git repository for code history, past and present issues and patches/pull requests/merge requests (complete with tags/labels, milestones, assignees, and similar metadata), past release announcements, and a wiki
- Project documentation and overview information for users and for developers, as would generally be available on a project's website (even if documentation source is available in the git repository, the snapshot we provide should include the docs as rendered into HTML)
- Archives of the project's mailing lists, as would generally be available as a Mailman, Majordomo, or similar instance, browseable and searchable via a web interface like this web view of python-dev (a Mailman 3 list)
Archives of the project's chat conversations, using a public Zulip chat history like this public archive of Rust's Zulip chat or a browseable history of Internet Relay Chat conversations via a web interface like this web view of Freenode's #pypa logs
(Olivier Lafleur conversed with me on Twitter about how to do some portion of this using GitLab. Also, the Perceval project may be a good tool to consider for mailing list and chat archives.)
Access privileges: The learner would not only interact with the example project materials as a reader but also as a participant, moving through the three access levels described below. The instructor would be able to view a learner's instance, with administrator (Level 3) access, to assess the learner's actions.
Level 1: The learner starts with the same user privileges that a new user would ordinarily have -- they can read all the public mailing lists, chat histories, wiki, and bugtracker items, and file bugs.
Level 2: The instructor can promote the learner to the second access level, at which point they can (for instance) triage, label, and close bugs and pull requests/patches, edit the wiki, and post to public mailing lists.
- Level 3: The instructor can promote the learner to the third (administrative) access level, at which point they can (for instance) browse and post to private mailing lists and chat channels, and use maintainer powers on the git repository (such as merging pull requests/patches).
Authentication: I imagine that dealing with authentication within the application could get sticky. My preferred approach:
- As a signed-in platform user, the learner is automatically logged in to all the web applications within the instance, and cannot log out.
- It is not possible to view an instance unless you are either the learner or instructor associated with that instance.
Multi-user access: Ideally, it would also be possible for an instructor to expand access so that a project can have multiple users (in other words, give Learner A access to Learner B's project instance), so that learners can engage in peer learning and group exercises. However, I expect that would lead to a much larger range of headaches, including Code of Conduct problems in interactions between learners, so -- in earlier versions of this tool -- I am fine with not having this functionality, and instead advising pairs and small groups to use screen sharing for group exercises.
End of life: At the end of the course, each instance would turn read-only for two weeks (to allow the learner to make notes, and to make local copies of any work they had done), and then the platform would delete the instance.
Thoughts? In particular, if you know of software that already does more than half of what I want, I'd like to know about it.
# 30 Sep 2020, 12:24PM: Changes Coming To Pip In October 2020:
People who deal with Python: Changes are coming to pip, Python's package installation tool, in October 2020. Please share this migration guide and our video with your circles.
I'm working on improving the Python packaging toolchain, foundational work that will (in the long run) make the whole Python experience way less confusing. In the short term this may mess with some people's workflows, so we want lots of people to hear about it now.
The pip team made a 2-minute video to explain what's up:
We are also doing user experience studies, and want you to sign up if you ever do anything with Python (whatever your level of skill/experience).
Please boost this toot or retweet this tweet if you want to help us get the word out.
Computers need to know the right order to install pieces of software ("to install x, you need to install y first"). So, when Python programmers share software, like when they publish packages on the Python Package Index or internally in large companies, they have to precisely describe those installation prerequisites. And then pip needs to navigate tricky situations when it gets conflicting instructions.
Up until now, pip's been very inconsistent in handling this stuff, which makes it easy for your Python environment to get messed up. That's why we successfully applied for $407K in funding from Mozilla and the Chan Zuckerberg Initiative to finish and roll out a proper dependency resolver for pip. The goal is that pip will get better at handling that tricky logic, and easier for you to use and troubleshoot.
You can test the new behavior (in beta) right now by using an optional flag in pip 20.2. And in pip 20.3, coming in October, the new behavior will be the default.
Once you're using the new resolver, pip is going to be stricter and more consistent. So things won't mysteriously break as much, and we can add more features that lots of people want.
But! Right now, a ton of people unknowingly have Jenga towers of wobbly dependencies in their environments and will run into pain when we make the resolver stricter and more consistent. And this may lead to you getting stuck in troubleshooting, assuming that pip caused the problem, when actually the deeper cause is conflicts among how your upstreams specify requirements (TensorFlow just fixed a related thing, for example).
So: We're trying to get Python users to try out the beta of the new resolver that's available in the current stable release of pip (20.2), fix your own environments, report bugs in your upstreams in advance, and report bugs to us so we can fix them in the next couple weeks. We started spreading the word about this a few months ago. And now: video! People watch videos, I hear? I hope this helps.
# 12 Nov 2019, 06:25PM: My New Title, Improving pip, Availability For Work, And SSL (No, The Other One):
A few professional announcements.
Seeking developers for paid contract on pip; apply by Nov. 22
One is that I helped the Packaging Working Group of the Python Software Foundation get funding for a long-needed improvement to pip. I led the writing of a few proposals -- grantwriting, to oversimplify -- and, starting possibly as soon as next month, contractors will start work. As Dustin Ingram explains:
Big news: the Python Packaging Working Group has secured >$400K in grants from multiple funders (TBA) to improve one of the most fundamental parts of pip: its dependency resolver. https://pyfound.blogspot.com/2019/11/seeking-developers-for-paid-contract.html
The dependency resolver is the algorithm which takes multiple constrained requirements (e.g. "some_package>=1.0,<=2.0") and finds a version of all dependencies (and sub-dependencies) which satisfy all the constraints.
Right now, pip's resolver mostly works for most use cases... However the algorithm it uses is naïve, and isn't always guaranteed to produce an optimal (or correct) result.
These funds will pay multiple developers to work on completing the design, implementation and rollout of this new dependency resolver for pip, finally closing issue #988.
Not only will this give pip a better resolver, but it will "enable us to untangle pip’s internals from the resolver, enabling pip to share code for dependency resolution with other packaging tooling". https://pradyunsg.me/blog/2019/06/23/oss-update-1/
This is great news for pip and Python packaging in general. Huge shout out to @pradyunsg for his existing work on the resolver issue and guidance here, and to @brainwane for all her tireless work acquiring and directing funding for Python projects.
If you or your organization is interested in participating in this project, we've just posted the RFP, which includes instructions for submitting proposals, evaluation criteria and scope of work.
If you're interested, please apply by 22 November.
NYU, Secure Systems Lab, and my new title
In further news: I am now a visiting scholar in Professor Justin Cappos's Secure Systems Lab at New York University's Tandon School of Engineering. And I get to use an office with a door, shelves, whiteboards, and so on (per the picture at right). If you contribute to Python packaging/distribution tools and live in/near or sometimes visit New York City, let me know and perhaps we could cowork a bit?
The Secure Systems Lab stewards The Update Framework (TUF) and related projects, and works to improve the security of the software supply chain. The Python Package Index is likely going to implement TUF to add cryptographic signatures to packages on PyPI, and so I've gotten to give TUF's developers some advice to help that work move along. (I won't be the manager on that project but I'll be watching with great interest.) PyPA may also choose to use more of SSL's work in implementing further security improvements to the package distribution toolchain, and I'm learning more to work out whether and how that could happen. Also, Cappos's research on backtracking dependency resolvers has been helpful to the pip resolver work.
Edited 19 Nov 2019 to clarify role.
I'm grateful to get to help connect the Python Software Foundation with more resources and volunteers. Changeset's current and recent projects have mostly been for the PSF. Last month we finished accessibility, security, and internationalization work on PyPI that was funded by the Open Technology Fund, and Changeset's work on communicating about the sunsetting of Python 2.x continues and will go through April 2020.
Availability for one-day engagements in San Francisco in February
But I am interested in taking on new clients for short engagements starting in February 2020. In particular, I will be in the San Francisco Bay Area in mid- to late February. If you're in SF or nearby, I could offer you a one-day engagement doing one of the following:
- developing a contributor outreach/intake strategy
- researching potential funders and writing a rough draft of a grant proposal
- auditing and improving your developer onboarding documents
I'd spend a little time talking with you, then sit in your office and finish the document before leaving that afternoon. (Photo at right provides a sample of how I look while sitting.) Drop me a line for a free initial 30-minute chat and we can talk pricing.
# 22 Sep 2019, 02:03PM: Futureproofing Your Python Tools:
The people who maintain Python and key Python platforms want to help you protect the code you write and depend on.
If you write software in Python, or depend on something that's in Python, this is for you.
Some of you are writing Python 2, or you still have software you depend on that is written in Python 2. January 1st, 2020, is the day that official support for Python 2 stops. So this is a fresh heads-up that you should really have a migration plan and start working on it, to move to Python 3. A lot of stuff you depend on already works on Python 3, and is even pledging to end Python 2 support in or before 2020. And it's easier than it's ever been to port your own code from 2 to 3.
You should probably upgrade to Python 3.7. If you want to test out 3.8, it has some changes in how it does warnings, and the first release candidate is out.
But, speaking of futureproofing:
Code authors move in and out of projects and companies. Six months or 18 months later, maybe you want to update and re-use, re-release or re-deploy code someone else wrote. Or you want your team to be able to reuse what you did after you leave, which means you need the code to run, and you need them to have the password so they can update stuff.
Have you written Python code that they have published as a package on the Python Package Index, pypi.org, so other people can use pip install to install it?
(And if you want to do that but don't know how? Check out this recently improved tutorial to help you do that.)
Publishing that package is a great way of making it so other people can run and deploy it, even within other parts of your organization.
But -- who actually has the keys to the castle? Who can upload a new version, or delete a version that has a problem?
You should probably make sure multiple people have either "owner" or "maintainer" privileges on the project on PyPI.
And you should review your project security history display, which lists sensitive events (such as "file removed from release version 1.0.1") in your PyPI user account and your PyPI project. We just added this display, so you can look at things that have happened in your user account or project, and check for signs someone's stolen your credentials.
And then how do you make it a little harder for vandals, spammers, and thieves to take over your account and upload malware or delete all the packages? Add two-factor auth for your login, like you would with your bank. Use an app on your phone to give you a six-digit code, or use a physical security device like a Yubikey.
And how do you make it easier to automate publishing new versions of your package, and make it safer to save your credential in the cloud? We've made it possible for you to create an "API token" where all it can do is upload, so you can use that instead of your PyPI password.
With well-tested Python 3 migration tools and new PyPI security features, now is a great moment to invest in robustness for Python software that you make or depend on.
[This blog post is kind of unwieldy, because it's about too many different things. I won't be publicizing it that much and instead will probably reuse text from it in more focused announcements elsewhere. But I'm publishing it here as a summary of my recent work, because management and communications for the projects above are what I've been working on recently for the Python Software Foundation. (A different kind of summary is on the Clients page for Changeset Consulting.)]
# (1) 08 Aug 2019, 11:49AM: When/How Do People Decide To Apply To CS Grad School?:
I'm trying to understand how people decide to go to CS grad school in the US. For instance, what proportion of PhD applicants are coming straight from undergrad, versus another graduate degree (such as an MS in math), versus industry? Do they generally decide first on what they want to research, whom they want to work with, or that they want to go to grad school at all? I presume there is a survey of this somewhere?
I've found the Computing Research Association's Center for Evaluating the Research Pipeline's report on why grad students choose computing, and I've also looked at the National Center for Science and Engineering Statistics website. I have found a little data on what proportion of doctoral recipients were previously in a baccalaureate program, a master's program, or industry, but not about what proportion of applicants, or
accepted applicants, come from those categories.
(Maybe there's paywalled research on this within the ACM's special interest group on CS education? If so, lemme know and I will try looking for that?)
The reason I am asking this is to help professors and guidance counselors I know (maybe you've heard that Outreachy has a new career advisor). They want to improve their abilities to help students and programmers consider research careers, and better target their outreach consider applying to specific graduate programs. How does the engagement funnel currently work? And thus, where are the gaps? I presume there are a bunch of people who would do well in grad school, and find it fulfilling, and use their research and their degrees in ways that would benefit the world, but no one ever says to them "hey have you thought about going to grad school," so they don't think of it as a possible thing for them.
# 22 Jul 2019, 11:35AM: Beautiful Soup is on Tidelift:
I've been doing a tiny bit of consulting for Tidelift for a little over a year now, mainly talking about them to open source maintainers in the Python world and vice versa. (See my October 2018 piece "Tidelift Is Paying Maintainers And, Potentially, Fixing the Economics of an Industry".) And lo, in my household, my spouse Leonard Richardson has signed up as a lifter for Beautiful Soup, his library that helps you with screen-scraping projects.
There was a period of about a year in 2017-2018 when I wasn't interested in doing Beautiful Soup work, but Tidelift changed that. Tidelift gathers subscription money from companies that rely on free software, and distributes the money to the developers in exchange for a level of support that I find sustainable.
Nobody builds an entire product around Beautiful Soup (or at least nobody will admit do doing this), but thousands of people have used Beautiful Soup to save time at their day jobs. Bundling Beautiful Soup together with bigger projects like Flask and numpy is a solution that works really well for me.
The other day I looked at the list of featured supported packages and was happy to see that a bunch of Python projects have signed up as lifters, including SciPy and numpy, Flask, setuptools, Werkzeug, websockets, urllib3, celery, coverage, a bunch of Django packages, Jinja2, keyring, Pylint, coverage, and pytest. And I think over the next 6-12 months we're going to see some effects of Tidelift support -- not just in the security and release cadences of the supported projects, but on other issues stemming from unfairness and a lack of reciprocity in open source, like maintainer burnout and expectation-setting. The list of licensing, security, maintenance, and marketing tasks lifters agree to do may end up being a benchmark, like the open source Independent Verification & Validation checklist by Open Tech Strategies, that even non-lifter maintainers use to set realistic expectations for what "supported"/"maintained" means.
# 03 Oct 2018, 05:16PM: Tidelift Is Paying Maintainers And, Potentially, Fixing the Economics of an Industry:
As the founder of Changeset Consulting, I keep my eye on consultancies and services in and near my niche, open source leadership, maintainership, and sustainability.* And I've known Luis Villa for years and got to work with him at Wikimedia. So yeah, I noticed when Tidelift announced its big new launch. And -- now, as a very-part-time consultant who helps Tidelift understand the Python world -- I am excited about their commitment to pay more than USD$1 million to maintainers (including "a guaranteed minimum $10,000 over the next 24 months to select projects").
Here's my take on the new Tidelift subscription model, the "lifter" role, and whom this works for.
For software businesses, this provides that missing vendor relationship, SLA, release cadence expectations, and general peace of mind for all of that unseen infrastructure you depend on. It's often easier for businesses -- of many sizes -- to pay a regular fee than to put open source project management work, dependency-updating, compliance checking, dependency security audits, or FLOSS volunteer relations on the engineering schedule.
For individual programmers and community-maintained open source projects, Tidelift is a potential source of substantial income. As a Pythonist, I hope to reach people who are currently core code contributors to open source projects in Python, especially on the Libraries.io digital infrastructure/unseen infrastructure/improve the bus factor lists. And I would like to reach projects like the ones Nathaniel Smith calls out in a recent post:
that (1) require a modest but non-trivial amount of sustained, focused attention, and (2) have an impact that is large, but broad and diffuse
and projects in the "wide open", "specialty library", and "upstream dependency" categories identified by the Open Tech Strategies report "Open Source Archetypes: A Framework For Purposeful Open Source".
For such people and projects, becoming a lifter is a promising model -- especially since the required tasks are fairly few, and are things maintainers should do anyway. I'm encouraged to see Jeff Forcier (maintainer of Fabric, Alabaster, and more) and Ned Batchelder's coverage.py getting onto the Tidelift platform.
And you can see estimated monthly income for your package right now. For some people, especially those whose healthcare doesn't depend on an employer, Tidelift payments plus some side consulting could be a sustainable, comfortable income.
Then there are folks like me whose contributions are only partially visible in commit logs (management, user support, testing, and so on), and groups that work together best as a team. Tidelift is also a potential source of income for us, but it's a little more complicated. Tidelift can send lifter payments to individuals, for-profits, and nonprofits, but: "If a package has multiple co-maintainers, you'll need to agree as a group on an approach." If you thought code of conduct conversations with your community were uncomfortable, wait till you bring up money! But, more seriously: I've been able to talk frankly with open source colleagues about thorny "who gets paid what?" questions, and if you're candid with your co-maintainers, the benefits may be pretty substantial. You can get advice on this conversation during the next live Tidelift web-based Q&A, Thursday, Oct. 11 at 2 p.m. Eastern Time (sign up at the bottom of the lifter info page).
Nonprofits, companies, and working groups that maintain projects can sign up now as lifters. Even if it's just a trickle of money right now, it might build over time and turn into enough to fund travel for an in-person sprint, contract work to improve continuous integration, an Outreachy internship, etc.
(One gap here: right now, Tidelift isn't great at supporting system-level packages and projects, like tools that get installed via apt or yum/DNF. I'm pretty sure that's something they're working on.)
What about noncommercial users or users who can't afford Tidelift subscriptions? The more lifters and subscribers sign up, the more those users benefit, too. Subscribers' funding means maintainers have time to make improvements that help everyone. And lifters agree to follow security, maintenance, and licensing best practices that also help everyone. Plus, Tidelift stewards libraries.io, a great resource for anyone who uses or develops open source (more on that). More money for Tidelift could mean libraries.io gets better too.
So I'm tooting a horn here and hoping more people sign up, because this is one of the more plausible ways open source sustainability could possibly work. Tidelift could be a real game-changer for the industry. Check it out.
* Examples: new competitors like Maintainer Mountaineer and OpenTeam, new funders like OSS Capital, and colleagues/referrals like Open Tech Strategies, VM Brasseur, Otter Tech, and Authentic Engine.
# 29 Jul 2018, 03:02PM: "Python Grab Bag: A Set of Short Plays" Accepted for PyGotham 2018:
Fresh from the waitlist onto the schedule: Jason Owen and I will be performing "Python Grab Bag: A Set of Short Plays" at PyGotham in early October. If you want to come see us perform, you should probably register soon. I don't yet know whether we'll perform on Friday, Oct. 5 or Saturday, Oct. 6.
(The format will be similar to the format I used in "Lessons, Myths, and Lenses: What I Wish I'd Known in 1998" (video, partial notes), but some plays will be more elaborate and theatrical -- much more like our inspiration "The Infinite Wrench".)
To quote the session description:
A frenetic combination of educational and entertaining segments, as chosen by the audience! In between segments, audience members will shout out numbers from a menu, and we'll perform the selected segment. It may be a short monologue, it may be a play, it may be a physical demo, or it may be a tiny traditional conference talk.
Audience members should walk away with some additional understanding of the history of Python, knowledge of some tools and libraries available in the Python ecosystem, and some Python-related amusement.
So now Jason and I just have to find a director, write and memorize and rehearse and block probably 15-20 Python-related plays/songs?/dances?/presentations, acquire and set up some number of props, figure out lights and sound and visuals, possibly recruit volunteers to join us for a few bits, run some preview performances to see whether the lessons and jokes land, and perform our opening (also closing) performance. In 68 days.
(Simultaneously: I have three clients, and want to do my bit before the midterm elections, and work on a fairly major apartment-related project with Leonard, and and and and.)
Jason, thank you for the way your eyes lit up on the way back from PyCon when I mentioned this PyGotham session idea -- I think your enthusiasm will energize me when I'm feeling overwhelmed by the ambition of this project, and I predict I'll reciprocate the favor! PyGotham program committee & voters, thank you for your vote of confidence. Leonard, thanks in advance for your patience with me bouncing out of bed to write down a new idea, and probably running many painfully bad concepts past you. Future Sumana, it's gonna be ok. It will, possibly, be great. You're going to give that audience an experience they've never had before.
# 18 Jul 2018, 06:58PM: Libraries.io and the Infrastructure of Hospitality:
So many times in my life in open source tools and platforms, I've run into the following problem:
We want to make a breaking change/prioritize work/get feedback.
Let's check with our downstreams.
How do we find and communicate with them?
We have stopgaps inside the application (like in-app and in-API messaging, Wikipedia banners) and outside (like creating users and announce mailing lists, publicizing social media broadcast venues, searching the web/GitHub for projects and people that mention/import our tool and pinging them personally).
Now, Libraries.io makes it way easier to be hospitable to my downstreams.
I co-maintain Twine, a utility that many developers use to upload packages to PyPI. We released 1.11.0 in March. The Libraries.io page about Twine shows me who's using it (see the screenshot on the left), and I can dive deeper to check who's pinned to an old version so I can ping them to check what's up. (Sometimes that conversation tells me about a problem in our upstream dependencies that I didn't know about.) If I want to survey some users to find out whether a particular breaking change would hurt them, this makes it way easier to find some representative users to ask, not just the power users and enthusiasts who take the initiative to reach out to me.
Libraries.io has an API so I could even automate some of this. And libraries.io covers npm, CRAN, PyPI, RubyGems, Maven, and a bunch more package managers across different languages and frameworks, so I'll see downstreams that aren't just in Python.
And the code is open source in case I want to understand how they rank projects in search results or add tox.ini support in their dependency checker.
I started drafting this post in March, and in the interim, I did a little paid work for Tidelift, the company that stewards libraries.io. So, disclaimer, I'm a bit biased now. But I thought libraries.io was supercool before Tidelift and I ever thought of working together. I'm glad the Ford and Sloan Foundations funded the initial version of this tool and I'm glad Tidelift is funding and using it now; it's already very useful, and I see it as part of the infrastructure that lets maintainers and users understand each other's needs.
# 11 Mar 2018, 10:42AM: Recent Debugging And Confidence:
I am proud of myself for some recent debugging I've done on and with codebases and tools that I hadn't worked on before.
A few weeks ago, I was sitting next to a friend who co-maintains a web app and hadn't looked at it for a while. The styling was screwy. I asked whether some CSS or JS he depended on had upgraded, like jQuery or something. He said no, his site hosted all its dependencies. I opened up the site and checked the Network tab in Firefox Developer Tools and saw that it pulled in Bootstrap from a CDN. Ah, one of the other maintainers had added that! And updates to Bootstrap had screwed up the page's styling.
That same day, as a freshly minted co-maintainer of twine (a utility to upload packages to PyPI), I investigated a problem with our CHANGELOG. Twine's changelog, as represented on Read The Docs (example) and when I built the docs locally, only displayed version number 1.4.0 (2014-12-12) and two associated GitHub issues. This was inaccurate since the source file changelog.rst had 70+ items and ran up to version 1.9.1 (2017-05-27). I figured out that this was happening because changelog.rst is meant to be formatted so the Sphinx extension releases (which I hadn't used before) can parse it, and the current file wasn't syntactically (or semantically) adhering to releases's conventions. (Since then, with advice and help from some folks, I've released Twine 1.10.0 and started a new maintainer checklist.)
And then, a couple days later, I fixed my friends' blog. Their front page had reverted to a ten-year-old index page. I had never touched Movable Type before and hadn't used their particular managed hosting web GUI before, but I poked around (and checked for backups before changing anything) and managed to figure it out: during a May 2008 outage, someone had hand-made an index.shtml page, which was now overriding the index.html page in the server config. I figured it out and found and fixed it.
My mom says that when I was a kid, I took apart alarm clocks and spare hose attachments and so on, and put them back together just fine. She once came upon me taking something apart, and when she drew breath to admonish me, I said, "Amma, if I don't take it apart, how do I know what is inside? Don't worry, amma, I'm just looking at it, I'll put it back together when I'm done," and I did. She told me that I took apart a mechanical alarm clock, carefully spreading all the parts out on some newspaper, and put it back together, and it didn't quite work properly, so I took it apart again and then put it back together, and it worked, and I jumped for joy and said "I fixed it!" (I still feel that way when I fix something.)
At some point along the way I feel like I lost that calm confidence in my abilities, that "things are made of stuff" and what one person made another can fix. But I have it again, now, at least for some bits of software, and some purely mechanical stuff (yesterday, helping friends move, deciding to break down a big empty cardboard box, responding to "but it's so big, it won't fit on the stack" with "we have knives"). It doesn't feel courageous at the time, just sensible, but then I look back and feel like a badass.
If I had to point to the single biggest cause of this regained confidence, I'd point to the Recurse Center, where I got way more comfortable with bravery and failure in programming.
# 06 Feb 2018, 09:48AM: The Ambition Taboo As Dark Matter:
PyCon just rejected my talk submission,* so I'll try to finish and post this draft that I've been tapping at for ages.
My current half-baked theory is that programmers who want any public recognition from our peers, recognition that meaningfully validates our personal mastery, basically have to do that through one of a few fora that therefore accrue less-spoken emotional freight. And two of those places are code review in open source projects** and proposal review in tech conference talk submissions, and the fact that we don't talk enough about the role of ambition when talking about these processes leads to unnecessary hurt feelings.
For context: We give talks for varied reasons. To teach, to make reusable documentation, to show off things we've made or things we know, to burnish our credentials and thus advance our careers, to serve our corporate brands' goals, to provide role models for underindexed folks from our demographics, to give a human face to a project and make it more approachable, it goes on.
A conference talk is a tool in a toolbox that has a lot of other tools in it. (The Recompiler, Linux Journal and LWN pay for articles, for instance.)
And conferences are more than lecture halls, of course -- they're networking opportunities, communities of practice, parties, vacations, sprints, and so on.
But when we talk about the particular pain or joy of having a talk accepted or rejected from a conference, there's an emotional valence here that isn't just about the usefulness of a talk or the community of a conference. We're talking about acceptance as a species of public professional recognition.
I've found it pretty useful to think about public professional recognition in the context of Dr. Anna Fels's book Necessary Dreams. She points out that the childhood or adolescent desire for fame is often a precursor to a more nuanced ambition, combining the urge to master some domain or skill with the desire for the recognition of one's peers or community. This influences how I think about awards, about job titles, and about encouraging technologists in the public interest, and about the job market's role in skill assessment.
So how can a programmer pursue public mastery validation? Here's what I see:***
- contributing to open source software (mastery validation: maintainers merging commits and thanking/crediting contributor for work)
- presenting at conferences (mastery validation: program committee accepting talk)
- posting comments to gamified platforms like Reddit, Hacker News, and Stack Overflow (mastery validation: upvotes and replies)
- publishing academic research (mastery validation: journal accepting paper, peers reviewing paper positively)
- writing books (mastery validation: publisher accepting & publishing book)
- starting and architecting technically challenging projects (mastery validation: skilled technologists cofounding with or working for you, or relying on or praising your work)
So, this stuff is fraught; let's not pretend it's not. And we get rejected sometimes by conferences and talk about it, try to take the perspective that we're collecting "no's", we remind others that even successful and frequent speakers get rejected a lot and you can choose not to give up. And we give each other tips on how to get better at proposing talks. And that's all useful. But there's also another level of advice I want to give, to repeat something I said last year:
I try not to say "don't get discouraged," because to me that sounds like telling someone not to cry or telling someone to calm down. It's a way of saying "stop feeling what you're feeling." Instead, I try to acknowledge that something is discouraging but also -- if the other person's ready to hear it -- that we can come back from that: your feelings are legitimate, and here are some ways to work with them.
Some advice I hear about bouncing back from a conference talk rejection involves formalizing, creating systems to use to get better at writing proposals (my own tips mostly fall into this category) -- after all, in programming, you can learn to make better and better things without directly interacting with or getting feedback from individuals. The code compiles, the unit tests pass. And that can be soothing, because you can get the feedback quickly and it's likely to be a flavor of fair. (But that computer rarely initiates the celebration, never empathizes with you about the specific hard thing you're doing or have just done, and rarely autocredentials you to do something else that has a real impact on others.)
To formalize and abstract something makes it in some ways safer; it's safer to say "I'm working to pass the [test]" or "I'm building a [hard thing] implementation" or "I'm submitting a talk to [conference]" than to say "I am working to gain the professional respect of my profession". But that is one motivation for people to submit talks to tech conferences and to feel good or bad about the talks they give.
So part of my advice to you is: go ahead and be honest with yourself about how you feel. Rejection can be hard, working to get an unaccountable gatekeeper's acceptance**** and failing to get public professional recognition in your chosen field is a cause of anxiety, and so on. Be honest about how discouraging that can feel, and why, and what you wanted that you didn't get.
And another part of my advice is that I will ask, like the annoying programmer I am: what problem are you trying to solve? Because there are probably a lot of ways there that don't involve this particular gatekeeper.
And the most annoyingly empowering part of my advice is: Humans created and run PyCon and TED and Foo Camp and all the other shiny prestigious things; you're a human and you could do so too. Especially if you acknowledge not just your own but others' ambition, and leverage it.
* Maybe we'll do it in an open space anyhow.
** Another blog post for another time!
*** I've left some things out here.
We have some awards, e.g., ACM Distinguished Member, that you might get if you work really hard for decades in certain fields. That feels too far away for the kind of thing I'm thinking about.
I've left out the possibility of being promoted at your job, because many technologists perceive engineering job promotions as not particularly correlating with the quality of one's work as a programmer, which means a promotion doesn't send a strong signal, understood by peers outside one's organization, of validation of programming mastery. Then again, if your organization is old enough or big enough, maybe the career ladder there does constitute a useful proxy for the mental models of the peers whose judgment you care about.
I've left out various certifications, diplomas and badges because I don't know of any that meaningfully signal validation of one's mastery as a programmer industry-wide. And there's a lot of stuff to parse out that I feel undecided about, e.g., I find it hard to distinguish the status symbol aspect of admission from the signal that the final credential sends. And: A lot of people in this industry find it impressive when someone has been admitted to certain postsecondary engineering programs, regardless of whether the person graduates. And: In my opinion, the Recurse Center is an experience that has an unfortunate and unintended reputation for gatekeeping on the basis of programming skill, such that a big subset of people who apply and are rejected experience this as an authoritative organization telling them that they are not good enough as programmers (and Google Summer of Code and Outreachy have a related problem).
Of course, go ahead, write your own blog post where you talk about how wrong I am about what I list or exclude, especially because I come from a particular corner of the tech industry and I'm sure there's stuff I don't perceive.
**** Some conferences' gatekeepers are more unaccountable than others'; regardless, the feeling from the rejectee's point of view is, I bet, mostly the same. And you can start your own conference or join the program committee of an existing conference to see what it's like from the other side of the desk and wield a bit of the power yourself.
# 02 Jan 2018, 10:46AM: 2017 Sumana In Review:
Four years ago, during my first batch at the Recurse Center, every day I'd write in a little notebook on the subway on my way home, jotting down a few bullet points about what I had learned that day. I found it helped in a variety of ways, and the keenest was that on bad days, reviewing my notes reminded me that I was in fact progressing and learning things.
On any given day in 2017 I often did not feel very happy with my progress and achievements and how I was using my time. I fell ill a lot and I was heartsick at the national political scene and current events. It is genuinely surprising to me to look back and take stock of how it all added up.
I went hiking in Staten Island and in the Hudson Valley. I got back on my bike and had some long rides, including on a canal towpath in New Jersey and over the Queensboro bridge. (And had my first accident -- a car in my neighborhood rear-ending me at a traffic light -- and thankfully escaped without damage or injury.) I learned how to bake bread. I got to meet Ellen Ullman OMG. And I tried to travel less than I had in previous years, but I still had some fine times in other places -- notably, I had a great time in Cleveland, I witnessed the total solar eclipse in Nashville, and I visited Charlotte, North Carolina (where, among other things, I visited the NASCAR Hall of Fame).
I did some of the same kinds of volunteering and activism that I'd done in previous years. For instance, I continued to co-organize MergeSort, participated in a fundraising telethon for The Recompiler telethon, signal-boosted a friend's research project to get more participants, and helped revitalize a book review community focusing on writers of color. Also, I served again as the auctioneer for the James Tiptree, Jr. Literary Award fundraising auction at WisCon, which is a particularly fun form of community service. The Tiptree Award encourages the exploration & expansion of gender. I wrote this year about what an award does, and the reflections I've seen from winners of the Tiptree Awards and Fellowships tell me those honors are doing the job -- encouraging creators and fans to expand how we imagine gender. This year I also deepened my commitment to the Tiptree Award by accepting the organization's invitation to join the Tiptree Motherboard; I am pleased to have helped the award through a donation matching campaign.
But the big change in my community service this year was that I tried to prioritize in-person political work. I called, emailed, and wrote postcards to various government officials. I participated in my local Democratic Club, including going door-to-door petitioning to get my local city councilmember onto the ballot for reelection.
And I found that I could usefully bring my technologist perspective to bear on the city and state levels, especially regarding transparency in government software. I spoke to my local councilmember about my concern regarding public access defibrillator data (the topic that led me to file my first-ever Freedom of Information Law requests, for government health department records) and this inspired him to sponsor a bill on that topic. (Which is now filed as end-of-session partly because of the limbo in potentially getting PAD data from NYC's open data portal -- I need to send an email or two.) I was invited to speak to a joint committee of the New York State Assembly on the software side of our forensics labs, and got particularly interested in this aspect of due process in our criminal justice system, publicizing the issue in my MetaFilter posts "'maybe we should throw an exception here??'" and "California v. Johnson". I testified before the Committee on Technology of the New York City Council on amendments to our open data law (I didn't prep my public comment, so this text is reconstructed from memory; video), and then spoke before the same committee on an algorithmic accountability measure (and publicized the bill, especially keeping the Recurse Center community apprised as best I could). And I did research and outreach to help ensure that a state legislature hearing on protecting the integrity of our elections included a few researchers and activists it wouldn't have otherwise.
In 2018 I want to continue on this path. I think I'm, if not making a difference, making headway towards a future where I can make a difference.
This was by far Changeset Consulting's busiest year.
I had a mix of big projects and smaller engagements. First, some of the latter: I advised PokitDok on developer engagement, with help from Heidi Waterhouse. For Open Tech Strategies, I wrote an installation audit for StreetCRM. And, working with CourageIT, I came in as a part-time project manager on a government health IT open source project so the lead developer could focus more on architecture, code, and product management.
Some larger and longer projects:
Following a sprint with OpenNews in December 2016 to help write a guide to newsrooms who want to open source their code, I worked with Frances Hocutt to create a language-agnostic, general-purpose linter tool to accompany that guide. "The Open Project Linter is an automated checklist that new (or experienced but forgetful) open source maintainers can use to make sure that they're using good practices in their documentation, code, and project resources."
I spent much of the first half of 2017 contracting with Kandra Labs to grow the Zulip community, helping plan and run the PyCon sprint and co-staffing our PyCon and OSCON booths, running English tutoring sessions alongside Google Summer of Code application prep, and mentoring an Outreachy intern, along with the usual bug triage, documentation updates, and so on. We wrapped up my work as Zulip's now such a thriving community that my help isn't as needed!
From late 2016 into 2017, I've continued to improve infrastructure and documentation for a Provider Screening Module that US states will be able to use to administer Medicaid better (the project which spurred this post about learning to get around in Java).
And just in the last few months I started working on two exciting projects with organizations close to my heart. I'm thrilled to be improving HTTPS Everywhere's project workflow for developers & maintainers over the next few months, working with Kate Chapman via Cascadia Technical Mentorship (mailing list announcement). And, thanks to funding by Mozilla's open source grants program and via the Python Software Foundation, the Python Package Index -- basic Python community infrastructure -- is getting a long-awaited overhaul. I'm the lead project manager on that effort, and Laura Hampton is assisting me. (Python milestone: my first time commenting on a PEP!)
Along the way, I've gotten a little or a lot better at a lot of things: git, bash, LaTeX, Python (including packaging), Sphinx, Read the Docs, Pandoc, regular expressions, CSS, the Java ecosystem (especially Gradle, Javadocs, Drools), the Go ecosystem, Travis CI, GitHub Pages, Postgres, sed, npm Linux system administration accessibility standards, IRC bots, and invoicing.
Talks And Other Conferences:
This year, in retrospect, instead of doing technical talks and expository lectures of the type I'm already good at, I played with form.
At LibrePlanet 2017 I gave the closing keynote address, "Lessons, Myths, and Lenses: What I Wish I'd Known in 1998" (schedule, video, in-progress transcript). I tried something aleatoric and it worked pretty well.
At Penguicon 2017 I was one of several Guests of Honor, and spoke in several sessions including "Things I Wish I'd Known About Open Source in 1998" (which was different from the LibrePlanet version, as intended) and "What If Free and Open Source Software Were More Like Fandom?" (further links).
Then, at PyGotham, Jason Owen and I co-wrote and co-starred in a play about management and code review: "Code Review, Forwards and Back" (video on YouTube, video on PyVideo, commentary).
I also attended Maintainerati and led a session, attended !!Con, worked a booth for Zulip at OSCON, attended PyCon and helped run Zulip's sprint there, and co-sponsored a post-PyGotham dinner.
Other Interesting Things I Wrote:
I did not write this year for magazines; my writing went into this blog, MetaFilter, Dreamwidth, microblogging, and client projects, mostly. I also wrote an entry for a local business competition (I didn't make it very far but I'm glad I did it, especially the finance bits) and started two book proposals I would like to return to in 2018.
I've mentioned already some of the posts I'm happy about. Some others:
"On Noticing That Your Project Is Draining Your Soul" (every once in a while someone emails me and mentions that this has helped them, which means a lot)
"How to Teach & Include Volunteers who Write Poor Patches" (12 things you can do)
"Inclusive-Or: Hospitality in Bug Tracking", a response to Jillian C. York and Lindsey Kuper.
I turned part of "Some posts from the last year on inclusion" into "Distinguishing character assassination from accountability", a post about pile-on culture and callout culture where I pulled out quotes from 11 writers on how we take/charge each other with responsibility/power within communities.
I loved Jon Bois's 17776 and discussed it with other fans on MetaFilter, and then, to try to understand its amazingness better, wrote "Boisebration", collecting links to fiction and nonfiction by Bois about class, feminism, aging, sports, politics, wonder, education, & art (and 17776 precursors/callbacks).
I found out about Robert E. Kelly, like so many did, when his kids crashed his BBC interview, then collected some links in a MetaFilter post about his writing on Korea, US foreign policy, international relations, and academia.
I wrote up a bit about "1967's most annoying question for women in Catholic ministry" on MetaFilter to signal-boost another Recurser's cool project.
I enjoyed the learning and the plot twist in "The programmer experience: redundancy edition", in which I discovered a useful resource for Form 990 filings and learned to use the Arrow library for Python date-time manipulation. And was grateful to Pro Publica.
And I made a few jokes on social media I particularly liked:
yesterday, was trying to explain virtual environments/containers/VMs to a friend and said "they range from Inception-style fake computers to putting a blanket on the floor and pretending it's lava"
today a friend and I explained leftpad & Left Shark to someone and I began sketching out a hypothetical HuffPo piece connecting them
We habitually crowdsource infrastructure from, expect unsupportedly high levels of performance from unsuspecting participants -> popcorn.gif
Public notice I received:
I got some public attention in 2017 -- even beyond the Guest of Honor and keynote speaker honors and my amazing clients -- that I would like to list, as long as I'm taking an inventory of 2017.
I rode the first revenue ride of the new Q train extension in Manhattan and really loved the art at the new 72nd Street MTA stop. A journalist interviewed me about that on video and my experience got into the New York Times story about the opening.
Presenters at the code4lib conference said their project was specifically motivated by my code4lib 2014 keynote "User Experience is a
Social Justice Issue" (written version, video). I was honored and humbled.
And -- this is out of place but I need to record it -- as someone who knew Aaron Swartz, I consented to be interviewed by artists working on a play about him, and so someone briefly portrayed me (as in, pretended to be me and repeated my words aloud) in that play, Building a Real Boy.
Finally, Hari Kondabolu looked at the English Wikipedia page about him, much of which I contributed, and was amazed at how thorough it was. So that was awesome and I was proud.
I got on Mastodon as part of my effort to improve how I use social media. I started using a new task tracker. I got back on my bike, and got somewhat into a habit of using it for some exercise and intra-city travel. A new friend got me into taking more frequent photos and noticing the world I'm in. Two new friends caused me to look for more opportunities to see musicians I love perform live.
I consumed a fair bit of media this year; didn't get into new music but enjoyed music podcasts "I Only Listen To The Mountain Goats" and "Our Debut Album". I did some book and reading reviews and will catch up to other 2017 reading sometime vaguely soon.
Leonard's film roundups & TV spotlights are a good way to see or remember most of what I saw in the last few years. TV highlights for me for 2017 are The Good Place, Jane the Virgin, The Great British Baking Show (which led me to write a tiny Asimov fanfic), Steven Universe, and Better Call Saul; I also saw Comrade Detective and Yuri!!! On Ice. Films I'm really glad I saw: The Big Sick, Schindler's List, Get Out (I fanned in MetaFilter Fanfare), In Transit, A Man For All Seasons, Hidden Figures, and Lemonade -- and a rewatch of Antitrust.
I made a few new friends this year, most notably Jason Owen and Mike Pirnat. My friends Emily and Kris got married and I got to hold up part of the chuppah for them. I took care of some friends at hard times, like accompanying them to doctor's visits. I got to see some friends I rarely see, like Mel Chua and Zed Lopez and Zack Weinberg, and kept up some old friendships by phone. My marriage is better than ever.
This year I shall iterate forward, as we all do.
# 03 Dec 2017, 01:23PM: The Programmer Experience: Redundancy Edition:
Sunday morning, one thought led to another, and I realized I'd like to read a US nonprofit's Form 990 (annual report on assets, revenue, etc.). I started looking for 990s, and stumbled upon a big set of machine-readable 990s from 2011 through sometime in 2016. Is the one I seek in there? How recently was the dataset updated?
So I broke out bpython, the json module, and wget and started figuring it out.
Some things I learned/realized/remembered along the way:
- json.load() does pretty much exactly what I hoped it would even though I had forgotten the exact wording -- yay.
- Gar -- why can't datetime Just Do The Right Thing regarding turning very datestamp-looking strings into datetime objects? Aha, Arrow is my answer (Arrow : datetime :: requests : urllib2).
- Hey, if I've started up a virtualenv (being careful to create one whose Python is my installation of Python 3), how come I can't invoke bpython? Ohhhh, I have to pip install bpython inside the venv (or -- just to be safe -- I used pip3 install bpython which worked).
- I am fine with iteratively searching through a huge dataset in an incredibly unoptimized and cut-and-paste way at first, and if I get irritated by waiting then I can fix things up with refactoring and optimization and so on.
- According to the index of available filings, the most recent update reflected in those filings is from January 11, 2017.
- I'm one step closer to reflexively doing this kind of stuff in a Jupyter notebook and publishing it but I'm not there yet, despite encouragement from Julia Evans and how much easier the infrastructure's gotten. Next time, I think.
- I remembered that Bradley Kuhn would like for far more people to pay attention to 990 filings, and has co-maintained a repository of FLOSS foundations' filings for some time. I started searching around in case there was a more recent version of the repo* -- and found, via an actually relevant textual ad served by a search engine, ProPublica's Nonprofit Explorer.
Use this database to view summaries of 3 million tax returns from tax-exempt organizations and see financial details such as their executive compensation and revenue and expenses. You can browse IRS data released since 2013 and access over 9.6 million tax filing documents going back as far as 2001.
For example, attractively presented data, and full 990s, for the Organization for Transformative Works.
So, uh, turns out I didn't need to actually get all "make a new GitLab repo! how does Arrow work? gotta refactor this!" and so on, but ah well. Now I will have a fresh new project to use for testing as I delve into Python packaging and test Warehouse, the new Python Package Index.
* Update: Mike Linksvayer, on Identi.ca, edifies: "The archived gitorious floss-foundations filings repo is now actively maintained by Martin Michlmayr at https://gitlab.com/floss-foundations/npo-public-filings".
# 27 Jun 2017, 02:52PM: Learning To Get Around In Java:
One of Changeset Consulting's clients is working on modernizing a legacy web application; we're improving both its structural underpinnings and its user interface and outgoing APIs. It is like we are Chief O'Brien in the first season of Star Trek: Deep Space Nine, surveying and retrofitting Terok Nor. But that's not a fair comparison; O'Brien has to not only grapple with alien engineering approaches, but with the resentful and deliberate trashing the Cardassians inflicted on the station before handing it over. I haven't seen Stargate Atlantis but perhaps that's a better analogy; with every component of this long-asleep lost city that we resuscitate, a new console or room shimmers to life. Which is pretty rewarding!
The original authors wrote this application in Java. I've never worked on a Java application before, so the last few weeks have been quite an education in the Java ecosystem, in its tools and frameworks and libraries. We're improving the installation and deployment process, so now I'm more familiar with Ant, Gradle, Maven, OpenJDK, JDBC, Hibernate, and WildFly. I've gotten some API documentation in place, so now I know more about Spring and Javadoc.
As I was explaining to a friend this weekend, the overwhelming thing isn't Java as a language. It is a programming language and you can program in it, fine. The overwhelmption is the seemingly endless chain of plugins, platforms, and frameworks, and the mental work to understand what competes with, supersedes, integrates with, or depends on what.
Imagine you come to visit New York City for the first time, and wish to visit a specific address. First you need to work out where it is. But you do not have a map; there is no unified map of the whole place. Surely you can figure this out. Watch out: if someone doesn't tell you what borough an address is in, it's probably in Manhattan, but then again maybe not. There are multiple streets with the same name, and "31st Street and Broadway" in Queens is quite far from "31st Street and Broadway" in Manhattan. The avenue numbers go up westwards in Manhattan, eastwards in Brooklyn, and northwards in Queens. And so on.
You ask around, you see sketches of maps other people have made on their journeys, and eventually you feel pretty confident that you know the rough distance and direction to your destination. Now, how do you get there from your hotel room?
You probably don't want to walk all the way; for one thing, it's illegal and dangerous to walk on the freeways. This is why we have the subway (express and local), and buses (express and local, both privately and publicly run), and government-regulated taxis (street-hailable cabs and private car services), and bike rental, and commuter rail, a funicular/tram, car rental, ferries, and so on. Also there are illegal rideshare/taxi services that lots of people use. You try to learn some nouns and figure out what sort of thing each is, and what's a subset of what.
A MetroCard works on some of these modes and not others, and some transfers from one ride to another cost you nothing, and you can't use an unlimited-ride card twice at the same station or on the same bus within 18 minutes.* You can bring a bike on some MTA-run services but not all, not all the time. There are whole neighborhoods with no subway service, and whole neighborhoods with approximately no street parking. At rush hour the trains get super full. Service changes at night, on the weekend, and on holidays. Cars and buses get stuck behind accidents and parades. People and signs in Manhattan refer to "uptown" and "downtown" as though they are cardinal directions; they often correlate to "north" and "south" but not always. Metro North trains terminate at Grand Central, but Long Island Railroad trains terminate at New York Penn Station, which is named after Pennsylvania because it's where you can catch a train to Pennsylvania,** and there's a Newark Penn Station too but over a crackling loudspeaker those two station names sound very similar so watch out. And so on.
You're lucky; you find a set of cryptic directions, from your hotel to the destination address, based on a five-year-old transit schedule. It suggests you take a bus that does not exist anymore. Sometimes you see descriptions of travel that you think could be feasible as a leg of your journey, and you read what other people have done. They talk about "Penn Station" and "the train" without disambiguating, refer to the subway as "the MTA" even though the MTA also runs other transit, talk about "the 7" without distinguishing local from express, and use "blocks" as a measure of distance even though some blocks are ten times as long as others.
Aaaaagh. And yet: you will make it. You will figure it out. New Yorkers will help you along the way.
The decades-old Java ecosystem feels overwhelming but this application overhaul is like any other task. Things are made of stuff. Human programmers made this thing and human programmers can understand and manipulate it. I'm a human programmer. I made Javadoc do what I wanted it to do, and now the product is better and our users will have more information. And every triumph earns me a skill I can deploy for other customers and groups I care about.
* Just long enough for you to enjoy a little break from the podcast you're making with President Nixon!
** Also see St. Petersburg's Finland Station.
# 15 Oct 2016, 01:55PM: New Zine "Playing With Python: Two of My Favorite Lenses":
MergeSort, the feminist maker meetup I co-organize, had a table at Maker Faire earlier this month. Last year we'd given away (and taught people how to cut and fold) a few of my zines, and people enjoyed that. A week before Maker Faire this year, I was attempting to nap when I was struck with the conviction that I ought to make a Python zine to give out this year.
So I did! Below is Playing with Python: 2 of my favorite lenses. (As you can see from the photos of the drafting process, I thought about mentioning pdb, various cool libraries, and other great parts of the Python ecology, but narrowed my focus to bpython and python -i.)
Playing with Python
2 of my favorite lenses
[magnifying glass and eyeglass icons]
by Sumana Harihareswara
When I'm getting a Python program running for the 1st time, playing around & lightly sketching or prototyping to figure out what I want to do, I [heart]:
bpython & python -i
[illustrations: sketch of a house, outline of a house in dots]
bpython is an exploratory Python interpreter. It shows what you can do with an object:
>>> dogs = ["Fido", "Toto"]
append count extend index insert pop remove reverse sort
And, you can use Control-R to undo!
[illustrations: bpython logo, pointer to cursor after dogs.]
Use the -i flag when running a script, and when it finishes or crashes, you'll get an interactive Python session so you can inspect the state of your program at that moment!
$ python -i example.py
Traceback (most recent call last):
File "example.py", line 5, in
toprint = varname + "entries"
TypeError: unsupported operand type(s) for + : 'int' and 'str'
[illustration: pointer to type(varname) asking, "wanna make a guess?"]
More: "A Few Python Tips"
This zine made in honor of
NYC's feminist makerspace!
CC BY-SA 2016 Sumana Harihareswara
Everyone has something to teach;
everyone has something to learn.
Here's the directory that contains those thumbnails, plus a PDF to print out and turn into an eight-page booklet with one center cut and a bit of folding. That directory also contains a screenshot of the bpython logo with a grid overlaid, in case you ever want to hand-draw it. Hand-drawing the bpython logo was the hardest thing about making this zine (beating "fitting a sample error message into the width allotted" by a narrow margin).
Libby Horacek and Anne DeCusatis not only volunteered at the MergeSort table -- they also created zines right there and then! (Libby, Anne.) The software zine heritage of The Whole Earth Software Review, 2600, BubbleSort, Julia Evans, The Recompiler, et alia continues!
(I know about bpython and python -i because I learned about them at the Recurse Center. Want to become a better programmer? Join the Recurse Center!)
# 26 Sep 2016, 09:33AM: iCalendar Munging with Python 3, Requests, ics.py, and Beautiful Soup:
Leonard and I love seeing movies at the Museum of the Moving Image. Every few months we look at the calendar of upcoming films and decide what we'd possibly like to see together, and put it on our shared calendar so we remember. And for every showing (example) the MoMI provides an iCalendar (.ics) file, to help you add it to your electronic calendar. But it's a pain to individually download or refer to each event's .ics file and import it into my electronic calendar -- and the museum's .ics files' DTEND times are often misleading and imply that the event has a duration of 0 seconds. (I've asked them to fix it, and some of their calendar files have correct durations, but some still have DTEND at the same time as DTSTART.)
Saturday morning I had started individually messing with 30+ events, because the MoMI is doing a complete retrospective of Krzysztof Kieslowski's films and I am inwardly bouncing up and down with joyous anticipation about seeing Dekalog again. And then I thought: I bet I can automate some of this tedious labor!
So I did. The create-fixed-ics.py script (Python 3) takes a plain text file of URLs separated by newlines (see movie-urls-sample-file.txt for an example), downloads iCalendar files from the MoMI site, fixes their event end times, and creates a new unified .ics file ready for import into a calendar. Perhaps the messiest bit is how I use a set of regular expressions, and my observations of the customs of MoMI curators, to figure out the probable duration of the event.
Much thanks to the programming ecology that helped me build this, especially the people who made RegExr, Beautiful Soup (hi Leonard), Requests, ics.py, and the bpython interpreter, and the many who have written excellent documentation on Python's standard library. Thanks also to Christine Spang, whose "Email as Distributed Protocol Transport: How Meeting Invites Work and Ideas for the Future" talk at Open Source Bridge 2015 (video) introduced me to hacking with the iCalendar format.
- It can be a bit slow as the number of URLs adds up -- it took maybe 5 minutes to process about 31 events. I oughta profile it and speed it up. But I usually only need to do this about six times a year.
- This script is not careful, and will overwrite a previously created .ics file at the same address (in case you're running it twice in one day). It has no tests and approximately no error-checking. This was a scratch-my-own-itch, few-hours-on-a-Saturday project. No Maintenance Intended.
- Absolutely not an official project of the Museum of the Moving Image.
# (1) 27 Jul 2015, 02:00PM PST: Slides & Code from HTTP Can Do That?!:
My slides are up, as is demonstration code, from "HTTP Can Do That?!", my talk at Open Source Bridge last month. I am pleased to report that something like a hundred people crowded into the room to view that talk and that I've received lots of positive feedback about it. Thanks for help in preparing that talk, or inspiring it, to Leonard Richardson, Greg Hendershott, Zack Weinberg, the Recurse Center, Clay Hallock, Paul Tagliamonte, Julia Evans, Allison Kaptur, Amy Hanlon, and
Video is not yet up. Once the video recording is available, I'll probably get it transcribed and posted on the OSBridge session notes wiki page.
I've also taken this opportunity to update my talks and presentations page -- for instance, I've belatedly posted some rough facilitator's notes that I made when leading an Ada Initiative-created impostor syndrome training at AdaCamp Bangalore last year.
# 18 Jun 2015, 06:53AM: HTTP Can Do That?! and Comedy:
On Wednesday of next week (June 24th) I'm presenting "HTTP Can Do That?!" at Open Source Bridge in Portland, Oregon.
I have explored weird corners of HTTP -- malformed requests that try to trick a site admin into clicking spam links in 404 logs, an API that responds to POST but not GET, and more. In this talk I'll walk you through those (using Python, netcat, and other tools you might have lying around the house).
I practiced this talk Tuesday night at the Recurse Center and it went well; people learned a lot about headers, verbs, status codes, and odd HTTP loopholes, and gave me constructive criticism so next week's version will be clearer.
I have also suggested a Birds of a Feather evening session called "Nothing Is Totally Incomprehensible If We Try Together" but don't yet know whether or when it will happen.
Then, at AlterConf Portland on Saturday, June 27th, I'll be performing some stand-up comedy for hippie nerds. I thought about trying to cram 100 punchlines into my 45-minute HTTP talk, but I don't think I'll be able to achieve that -- people need to understand something before they can understand a joke about it -- so it'll be nice to get 4 or 5 laughs per minute during the stand-up on Saturday.
# (1) 24 Apr 2015, 03:26PM: Technothriller Book Review Partially In The Form Of A Python Exercise:
I am glad I read Hackster: The Revolution Begins..., a technothriller by Sankalp Kohli and Paritosh Yadav taking place in modern-day India. It's plotty and passionate and tense, and it's about Indians to whom India is the center of the universe. But it's also got major problems. Here are some quotes:
It was now time to attain answers. And he had found his answers in SNAGROM -- a device conceptualized by his father, but built and made operational by him with a few modifications to avenge the death of his patriotic father who had sacrificed his whole life for the progress of beloved country, India, only to be publicly humiliated and pronounced a terrorist with links to Pakistan's ISI by the ruling party of India, The Democratic Alliance Party. [p. 23]
Mr. Bedi, Vikram's father, was a scientist. He had the unique ability to solve problems by using concepts of one domain, into an altogether different one - something which most academicians couldn't do. His papers and theories on early meta-systems had brought a fresh perspective and direction into the scientific community. In his papers, he reduced the bigger problems into simple ones. He put it very simply, a meta-system is a system based on other systems. [p. 35]
Arjun could feel this guy getting to him.... he was not a person who took even the smaller defeats sportingly. For him defeat was accompanied by a splurge of vengeance. [p. 68]
"It seems like he had conceptualized a system that replicated the modern day concept of Big Data trackers and used it to come out with trends which were closer to reality." Vikram whispered to himself. [p. 78]
But, was it all because of one man? How could a single man cause so much havoc? It must have been 'the system'. [p. 111]
For ten years, he had used his peculiar ability to suppress all sorts of mutiny within the alliance with an ease that always surprised everyone around him. Nobody had ever seen him running across the country to meet the influential people in times of crisis. He would simply make a private phone call and follow up the next day. The matter would be resolved. [p. 152]
So I didn't love the prose or the characterization. And one plot thread in Hackster disproportionately bothered me.
In the scene below, two guys are investigating a break-in by Vikram, a super-elite hacker. Vikram broke into the Srinagar police department's "criminal database" to remove his friend Ashfaq's name from "the list of arms dealer with a pending investigation" (sic). Initially, police investigators had overlooked the incursion: "They termed it a routine hack failure." [p. 17-18] But this new anti-cybercrime unit digs deeper. For context, both authors of Hackster have MBAs, one "in the field of telecom technology," and in the Acknowledgement they thank someone for cybersecurity advice.
"He deleted one entry and then used a jumbler on all the others."
"After deleting the entry, he covered his track by jumbling up the names of all the people in the list. I tried running a point to point match between the shuffled copy of this list with an older correct copy, but none of the names matched. In short the whole list is corrupted, and we will not be able to make anything out of it easily. It is a long list. It has too many names. This guy is a genius." [p. 51-52]
But then Aarti, a top-shelf cybersecurity expert, succeeds at extracting the name "Ashfaq Ahmed Karim":
"He didn't know that entire data of servers of police department gets automatically stored in tape drives at the end of each month. These tape drives are detached from the servers and are stored in a secret location. I took out an older version of Illegal Arms Dealer List from the backup tape drives and then wrote a program to match each word of the older list with the newer one and rearranged the new list accordingly."
Sumit and Rao watched her with awe as she continued further, "Even the most advanced computer of ours took two days to complete this activity and give us this one name. This one lead should help us to take a step closer to our target." [p. 82]
My suspension of disbelief at this point broke so hard that it sent shards into nearby brick walls, where they remain, softly vibrating. I'm willing to set aside, for the sake of fiction, how badly guarded this data is, and why does Aarti have to go to the tape drive if there's an older version of the list more readily available, and why are they acting like this is a giant string rather than a set of rows in a table in a relational database and thus amenable to additional forensic techniques. Even so: this kind of puzzle is practically a junior programmer's intro-to-Python exercise. You could do this in bash; you could do it in Excel. And unless the Srinagar police department is tracking pending investigation against literally millions of arms dealers, a bog-standard developer's laptop could run that script in, mmm, 20 minutes.
Hmmmmmmmm, how long would it actually take? I decided to try to replicate this, without even trying very hard and while listening to a Taylor Swift album on repeat. I took the 417 names from the Nielsen Haydens' old blogroll, put them into a file separated by newlines (bloggers-archive.txt), and then removed one name, and saved the new file as bloggers.txt. Ah but now I want to obfuscate it! So I pulled all the names apart into their component words and shuffled them randomly and then wrote that back to a file (code: obfuscate.py). The new, jumbled list looks suitably forbidding:
My findmissingname.py script does not bother to "rearrange the new list accordingly" because what Aarti really wants is the missing name. findmissingname.py spits out the two words in the missing name, and it takes 0.04 seconds to do so on a ThinkPad. And I'm bone certain I could optimize performance further.
This points to an asymmetry I had not previously noticed regarding what will and will not break my suspension of disbelief. When I'm reading scifi or technothrillers, I am reasonably fine with magic zoom-enhance, encryption, robotics, and other implausible advances. I can deal with it if you have way cooler toys than exist in my world, if you tell me something hard for me is easy for you. But if you try to tell me that something easy for intermediate-skilled me is hard for hella competent world-class experts with best-of-breed gadgets, I laugh, because you're ridiculous.
I am married to a programmer whose code has literally been used to catch an illegal arms dealer. I highly doubt this repository is going to have a similar impact. But hey, I learned something new about my genre reading conventions and I practiced my Python 3.
# 19 Jan 2014, 04:51PM: Interesting-Looking Talks at PyCon 2014:
This year I'm going to visit PyCon! In fact, I'm presenting a poster: "What Hacker School Taught Me About Community Mentoring". You should register soon if you're coming, especially to take advantage of heavily subsidized childcare or to register for one of the tutorials.
Someone on one of my mailing lists asked what sessions people are particularly looking forward to. I tend to follow Skud's conference tips, which mean skipping sessions when I need to do self-care. But with such great-sounding talks, I may not be able to pull myself away!
- Allison Kaptur's "Import-ant decisions". Kaptur is a facilitator at Hacker School and I enjoy her thinking
process, areas of interest, and speaking style. I know I'll learn more about package management in general, and about Python specifically, from this talk.
- Jessica McKellar's "Building & breaking a Python Sandbox". McKellar did a residency in my Hacker School batch during which I got to see a preview of this talk, so I may not go again, but I found it thought-provoking; it helped me understand how Python works in a new way.
- Erik Rose's "Designing Poetic APIs". I met Erik Rose at a Google Summer of Code Mentor Summit and thought he was a really smart and fun guy. I've never designed an API before (thus I also want to go to the novice-focused "So You Want to Build an API?" by Megan Speir), and I like that his discussion will include anthroplogy, psychology, and history.
- Nathan Yergler's "In Depth PDB". I am one of the programmers the speaker sadly mentions; I rarely use it, if at all, even though it would be great to help me see when and why things are breaking!
- A localisation talk, maybe this one, because I think the Python application I'm working with right now just has hardcoded strings (aiee, bad).
- One of the SQLAlchemy talks, maybe this one, because I don't grok how to use SQLAlchemy yet. However I have registered for the SQLAlchemy in-depth tutorial so one may duplicate the other.
- Julia Evans's "Diving into Open Data with IPython Notebook & Pandas".
I enjoy Julia Evans's investigation process and her speaking and writing style. (See my post "Why Julia Evans's Blog Is So Great".) I have never used IPython Notebook nor matplotlib, numpy, pandas, or any of the other awesome science/data-related Python tools, and keep meaning to; this talk should help me with that.
- Greg Wilson's "Software Carpetry: Lessons Learned".
I am a tremendous fan of Wilson's work - Software Carpentry, the books he's edited, etc. The SC crowd has collected a lot of data (e.g. surveys of learners at their bootcamps) and I will probably want to soak in their lessons learned and shout about them to every other teach-y group I'm in.
- Naomi Ceder's "Farewell and Welcome Home: Python in Two Genders". I want to learn more about the experiences of trans women in my open source communities.
- Kate Heddleston, Nicole Zuckerman, presenting "Technical on-boarding, training, and mentoring". I do this task in my open source communities so I want to learn more best practices.
I'm thoroughly looking forward to my first PyCon. (I stopped by one for like an hour in 2003 and helped at the registration desk; I guess it took me eleven years to get to the other side of the desk!)
# (2) 22 Dec 2013, 10:42AM: Why Julia Evans's Blog Is So Great:
Some writing is persuasive; it aims to cause you to believe or do something. Some is expository; it aims to cause you to understand something. A lot of tech writing is persuasive or expository.
Some writing is narrative. It aims to cause you to feel or experience something. In personal narrative, the writer shares a personal experience and invites you to walk with her on that journey, experiencing it as she did, emerging with a new perspective. I really like narrative-style tech writing.
What I call the "Amazing Grace" story (previously) is, in a sense, all three of these. "Amazing grace! (how sweet the sound) / That sav'd a wretch like me! / I once was lost, but now am found, / Was blind, but now I see." Or, in more modern terms, "An English Sailor Found Salvation Through This One Weird Trick."
- Exposition: My experience started in sordid terror and ended in divine ecstasy
- Narration: Bask and wonder with me in the intricacy of my journey and the unexpected yet inevitable emergent properties of my condition
- Persuasion: Thus, if you are enthralled to sin, if you are a fallen resident of our fallen world, you should follow my example
I started thinking about this because my Hacker School colleague Julia Evans has a super-engaging blog. During our batch, she dove into operating system internals, and blogged about what she learned and how she learned it. She's consistently inspired me and made me laugh. Two of her fans (fellow HSers) even made a loving Markov-chain tribute, Ulia Ea.
One reason we love it is that most entries narrate her daily learning and illustrate a journey through confusion into wonder. See "Day 37: After 5 days, my OS doesn't crash when I press a key", which is possibly the most "Amazing Grace"-esque of her posts. Excerpt:
5. Press keys. Nothing happens. Hours pass. Realize interrupts are turned off and I need to turn them on....
It's not just the large-scale rhetorical structure; her diction and even her punctuation delight me. I particularly marvelled at her sentences in "Day 43: SOMETHING IS ERASING MY PROGRAM WHILE IT’S RUNNING (oh wait oops)". Excerpt:
12. THE OS IS STILL CRASHING WHEN I PRESS A KEY. This continues for 2 days....
As far as I can tell this is all totally normal and just how OS programming is. Or something. Hopefully by the end of the week I will get past "I can only receive one IRQ" and into "My interrupt handler is the bomb and I can totally write a keyboard driver now"....
I'm seriously amazed that operating systems exist and are available for free.
SURPRISE MY CODE IS NOT WORKING BECAUSE SOMETHING IS ERASING IT.
Can we talk about this?
- I have code
- I can compile my code
- Half of my binary gets overwritten with 0s at runtime. Why. What did I do to deserve this?
- No wonder the order I put the binary in matters.
It is a wonder that this code even runs, man. Man.
The disarmingly informal ALLCAPS adds to the intimacy more explicitly created with the question "Can we talk about this?" which invites the reader into one-on-one conversation. Moreover, I specifically call your attention to the statement "Why." and the repetition "man. Man." They demonstrate how Julia acknowledges mystery, with a tinge of disbelief.
As Patrick Nielsen Hayden observed,
A great deal of science fiction is about what the field's insiders often call "sense of wonder," a quality not entirely unrelated to the good old Romantic Sublime. Many of the genre's classics are in essence carefully-tuned machines designed to attract readers whose primary conscious loyalty is to rationalism, and lead them by a series of plausible contrivances to a sudden crescendo of mystical awe. This is an important part of SF from Olaf Stapledon to William Gibson and beyond.
And Julia Evans.
You can hire me through Changeset Consulting.
This work by Sumana Harihareswara is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.
Permissions beyond the scope of this license may be available by emailing the author at firstname.lastname@example.org.