# (0) 05 Dec 2013, 08:51AM: Fisher-Price's My First Twitter Bot:
On Sunday I wrote my first Twitter bot, with a bit of help from Leonard. (A Hacker School colleague inferred, understandably, that Leonard and I just write Twitter bots on the weekend, to relax.) Then yesterday I helped a peer get her first Twitter bot going, and decided to write it up for y'all/future Pythonistas.
already written other guides to writing good Twitter bots. Those more experienced people are better than I am at writing good, surprising bots, ones that respond to tiresome hashtags or argue with followers or what have you. My guide, in contrast, assumes that you already have some kind of code that spits out strings of 140 characters or fewer, and just want help interfacing with the Twitter API using Python so a Twitter account can say those tweets.
I am posting this guide on 5 December 2013, and undoubtedly Twitter will revamp all this within the next fortnight and switch to OAuth 2038 (OAuth For Workgroups) and turn their developer site into something you manipulate by holding up your mobile phone in front of your Google Glasses. But for right now this works.
- I rely on the python-twitter wrapper around the Twitter API. If you are using a virtual environment that you made with mkvirtualenv or similar, then activate your venv and
pip install python-twitter
If you live on the edge, or if you are a devil-may-care type who loves convenience so much more than safety that you tab-complete your passwords, then command-tab away from Snapchat* long enough to install the package globally:
sudo pip install python-twitter
- Now, go to Twitter.com and create a Twitter account for your bot, e.g., @FreedomBot. Make sure that you give Twitter an email address whose messages you can actually read. Read the end user license agreement and agree to it. Did you know that there's a hotel in San Francisco named the Eula?
- Look for the automatic email from Twitter and follow the instructions to confirm your account. Once you've done that, go to dev.twitter.com and sign in with the username and password of your bot's Twitter account, e.g., your username is FreedomBot.
- Go to https://dev.twitter.com/apps and choose to create a new app. Yes, your "Markov chain parody of the Office of Management and Budget" script is an "app" for our purposes; this is the easiest way to get the relevant keys.
- Go ahead and fill in a reasonable name, description, and link (your website or Gitorious repo would do). These will not be publicly visible or scrutinized by Twitter's gatekeeper gnomes, so don't sweat it. Leave the "callback URL" field empty. Read the Rules of the Road (so much more folksy than "Guidelines for the Walled Garden") and agree.
- Now that you have an "app", go to the Settings tab and change your app type from Read-only to Read/Write, so you can actually post to Twitter from your script. (If you're having trouble navigating back to your app, go to dev.twitter.com/apps to find it.)
- Next, on your app's Details tab, click the button at the bottom to create your API secret key & token. It'll take a minute for Twitter to create those for you; after a minute, go ahead and refresh the page. Now that page gives you the "consumer key" and "consumer secret" as well as the API "access token key" and "access token secret".
(The "consumer key/secret" pair identifies your "app" (the code you are writing) as something that is allowed to interact with the Twitter API; it's sort of a substitute for the User-Agent string in a browser. The "access token key/secret" pair authorizes the Twitter account whose tweets the "app" is gonna write. So conceivably you could write an app like Sycorax that lets your bot roleplay multiple accounts, and it would end up getting multiple access token key/secret pairs so that could work. This is all OAuth stuff that briefly turns you into an octopus when you understand it.)
- So, how can your script use these secret strings (to authenticate and authorize you) without those supersensitive nuclear launch codes falling into the wrong hands, viz., your GitHub repo? I did it this way:
Open up a new file in the same directory as your bot script, called something like "twitterapi.py". Also add "twitterapi.py" to your repository's .gitignore.
#!/wherever/you/put/python # ok, probably /usr/bin/python
api = twitter.Api(consumer_key="KEY",
So now, in your application or in the Python interpreter, you can do:
from twitterapi import api
- And then, to post your awesome string to Twitter as @FreedomBot:
api.PostUpdate("wow\n so Program Assessment Rating Tool\nnice\n very budgetary")
- Optional step! If you want to leave your bot running to tweet at 5-minute intervals, then
import time and stick a
time.sleep(5*60) line and an
api.PostUpdate(awesomestring) into a
while True loop.
* Is there a Snapchat desktop app? Maybe Snapchat is available for the iPad and in my hypothetical you're programming on your tablet? I don't know what people do these days.
: Hacker School
# (0) 30 Nov 2013, 06:37PM: Round Seven of OPW:
Congratulations to all six of the Wikimedia's chosen participants in the current round of FOSS Outreach Program for Women internships. I'm especially glad I was able to help Maria Pacana and Be Birchall, my colleagues via Hacker School, learn more about the program and apply.
Many months ago, Wikimedian Liam Wyatt tweeted:
@hexmode @brainwane prob a dumb question, but do we have anything planned like @codeacademy to help folks learn mediawiki/php/wikimarkup?
The answer: now we do. One OPW intern will make a Codecademy course on the MediaWiki API. (Also, we now have The Wikipedia Adventure and the Visual Editor to help people start editing.)
# (0) 30 Nov 2013, 11:11AM: Colons: A Retrospective:
I am working on another silly project, and it led me to look at winners and finalists for the Pulitzer Prize for General Nonfiction. There are way more colons in the titles of those books than there used to be.
At Leonard's suggestion, I did a stacked graph via LibreOffice.
Here's the original data, and here's the breakdown by year. I went from 1980 to the present because 1980 is the first year the Pulitzer folks released the list of finalists.
I know many people will not find it surprising that we've got a whole lot of colons these days. But look at the old titles: there used to be so few! 1963: Barbara W. Tuchman's The Guns of August. 1975: Annie Dillard's Pilgrim at Tinker Creek. I may ask some publishing-y people what forces and trends change book titles over time, and whether Pulitzer finalists are outliers when it comes to book-naming. Three or four titles per year makes for a ridiculously tiny sample size.
# (1) 27 Nov 2013, 03:45PM: Shiny:
I hereby recommend to you the super-readable, witty, on-point analysis of cosmetics ad claims at "Brightest Bulb In the Box: Beauty for Critical Minds". Much thanks to terriko for the link to BBItB! If you liked Constellation Games, you might imagine Robyn as a genderswapped Ariel Blum. If the aliens show up, she may demand to try and test their cosmetics. I had no idea I wanted to read beauty blogging until I came across Robyn.
I love her perfume reviews, e.g.:
This is the most generic perfume ever. Like, if you didn't care about perfume and just sort of imagined something boring, this is what it would smell like.
If this scent were being worn by a fictional character, it would be Ann Veal from Arrested Development.
Robyn also makes her research available free-as-in-blush, e.g., testing "What Methods of Foundation Application Use the Least Product?" or "How Much Do Your Eyeshadow Brushes Matter?". Most recently, she got out the chi-square to compare two different monthly subscription boxes a few different ways.
But I especially want you to check out her resveratrol and Urban Decay Naked Skin Beauty Balm posts. Her commentary on "light-defusing spheres" especially made me guffaw. Other tidbits:
"DNA repair, optical blurring, oil free"? One of those claims just doesn't belong. (And it is the last one, because it makes sense.)...
First, I want to deal briefly with "reseveratrol". Juice Beauty spelled the name of their supposed active ingredient incorrectly. What they mean to say is resveratrol, which is a phenylpropanoid that is found in the skins of grapes....
If you are a yeast cell, congratulations on your literacy. Maybe check out this resveratrol thing. If you are a human, though, you should know that at the present time, there are NO peer reviewed journal articles that suggest that resveratrol has any effect on people....
Also, Robyn's leitmotif "your face" (e.g., "Your imperfections really would be less noticeable in diffused light, but the solution to that is to avoid uncovered bulbs in your house, not to put this stuff on your face.") reminds me of Danni, who says "your face" a lot and whom I miss.
# (0) 25 Nov 2013, 04:31PM: The Last One You'd Ever Suspect:
Check out this Vienna Teng live set in which she performs a synth-backed "Whatever You Want." I'm just entranced and have been listening to the set over and over. I especially find myself caught by the line
I am the last one you'd ever suspect of setting the fire, of setting the fire
# 23 Nov 2013, 01:59PM: I Cannot Be The First Person To Quip About Quantified Self-Loathing:
After the first week I spent at Hacker School, I worried that I wasn't spending enough time on improving my programming skills. So I started using Project Hamster to track chunks of time that I specifically spent either learning (via coding, pairing, or listening to useful lectures, mostly), versus chunks I spent teaching or helping others.
This past week, I looked at my involvement with Bicho, an open source project that helps people analyze data from bug trackers, and decided there were too many blockers for me to keep on going as I was going. Thriving is a function of a person times their environment, as I learned in my tech management courses, and -- as I wrote in a summary on the metrics-grimoire mailing list -- at my current level of programming proficiency, and given how much refactoring and testing Bicho could use, it's just a bad fit right now. The maintainers responded well, and promise a refactored Bicho is coming, so I hope to restart contributing at some point in the future.
I wondered, after I stopped: how much time had I spent on this project, and what had I learned from it? So I crunched the numbers. Between October 7th and today, I've spent 158 hours on learning activities and 12.9 on teaching/helping activities, which gives me 173.2 hours in total. (I was sometimes rough when inputting my time into Hamster, so take my significant digits with a grain of salt.) Of those, I've spent 55.9 on Bicho, 53.9 on learning and about two on teaching/helping (such as filing bugs and writing that super long email). So that's a little under a third of my Hacker School learning time.
What did I learn? I threw together a rough list:
- read a big giant codebase and understood how parts work together
- got lots more experience with git
- learned how argparse works enough to port something to it from optparse
- used XML-RPC APIs, xmlrpclib
- used launchpadlib module
- learned how packaging works (setup.py, pip)
- grokked __init__ and modules vs packages
- learned some of PEP 8 and used the pep8 and pep8ify modules
- used assert & wrote tests
- used Beautiful Soup
- used some of vars & getattr
- used pdb a little bit
- learned a little bit re Storm and ORMs
- used MySQL & python-MySQLdb
- learned to watch out for old- versus new-style classes
- got comfortable with virtual environments
- wrote a regex
- learned not to put an unquoted argument on the commandline (ampersands!)
That first one is huge. I think it may just take a super long time the first time you try to wrap your head around a codebase fifteen thousand lines long. Then again, now that I've had this experience, I've ordered Michael Feathers's Working Effectively with Legacy Code and may start following Jessica McKellar's advice to Maria Pacana: [Don't] try to understand the whole thing. Understand only as much as you need to know to make the contribution you want to make.
I've now moved on to a different project where I'm making clearer progress, though sometimes it's a slog. In retrospect, I don't really know whether my Bicho work was a good investment of my Hacker School time, or whether I should have stopped a few weeks earlier and learned more and different things. I am trying to remember not to fall prey to the fallacious Fear Of Missing Something. Maybe part of what I learned is a better intuition for "it's time to try a different approach." Argh. So hard, maybe impossible, to assess whether I made good decisions!
: Hacker School
# 23 Nov 2013, 11:40AM: How Comprehensive Are Your Unit Tests? Coverage.py Knows:
I've been writing and maintaining unit tests for my project. But only on Thursday did a colleague's presentation remind me that I could run a code coverage tool to check which code paths my tests are or aren't exercising.
I found it super easy to install and run coverage.py, and it only took marginally more fuss to
--omit="~/.virtualenvs/*". The detailed feedback helped me increase my coverage from 70% to 82%; yay! Thanks, Ned Batchelder & other coverage.py contributors.
: Hacker School
# 16 Nov 2013, 10:58PM: A Little Design Thinking Can Go A Long Way:
I was playing with stdin/argv because Leonard suggested I improve
Missing from Wikipedia to make it more Unixy and interoperable with other scripts and systems present and future. Right now it demands that you tell it the name of an existing plaintext file as a positional argument. Why shouldn't you be able to generate a giant string of names separated by newlines and just pipe it into the script, as you would into
grep, and similar tools?
I struggled with this whole stdin business, trying to make the tool work with both types of data input, and became disheartened. Then I stepped back to think about what I actually want to do. Aha: I am facing a design decision. I could make different choices that would suit different audiences.
For context: I took a rhetoric class in 1998 and learned the classic Rhetorical Triangle governing any communication. I then misremembered it for more than a decade till I looked it up just now. But I like my version better. So! Sumana's Rhetorical Triangle, as applicable to a piece of political software as it is to an essay, says that if you are trying to communicate with someone, it helps to consider:
My message: some topics have way less coverage on the Wikipedias than they deserve. I feel fine sticking with that. But who are my audiences, and thus which medium should I choose?
If I want terminal-savvy researchers and developers to use this tool, then it's fine as a standalone command-line script. I should stick a
setup.py in there and put it up on PyPI, and switch to an all-stdin model of data input.
If I want activists and less programming-savvy researchers to use it -- people not like me -- then the path gets foggier. I haven't tested this script on a Mac or on Windows; I could work to make sure it's friendly on those OSes, and stay with the simple "gimme a textfile" data workflow. (Why make my user learn to use pipes and
But the much user-friendlier step would be to turn it into a little web app on Tool Labs. My tool would read input from a bunch of formfields and/or allow the user to upload a CSV-type file, and could output to a nice-looking HTML page with redlinks (to help you create the pages) with options for plaintext or wiki markup download. This would also make the tool a lot more discoverable by casual websurfers. And if I put it on Tool Labs, I can run queries directly against live replicas of the Wikimedia databases, which would be faster than hitting a web API.
I imagine some folks, who like great UI and more seamless data transfer, would prefer installable desktop/mobile applications with actual GUIs. But I have approximately no skills in that area and feel very little urgency about growing said skills, so I won't be going in that direction.
Once I framed my data flow problem more as a product management question and less as an implementation struggle, I found it much easier to decide. I can serve the audience that needs this tool -- activists and researchers -- while still retaining value for those with more comfort on the command line. It would be feasible to refactor the tool into:
And I've not yet implemented a web app that takes input from a user and spits out a relevant response, so I could do that and become a cleverer programmer, or borrow code that does most of what I want.
- a core module that takes a bunch of names, checks them against a Wikipedia, and spits out a "missing" list (you could run this as a standalone command-line script, getting data from stdin)
- a set of web-specific functions that make it easier to get input and excrete output
The simplification that makes me sigh in relief: I won't write and maintain two kinda-clashing methods of data input. (Although the tradeoff is a bunch of (arguably) feature creep.)
: Hacker School
# 16 Nov 2013, 09:45PM: Accidental Quine:
On Friday, while trying to work with standard input (stdin) and command-line arguments (argv), I accidentally wrote an almost-quine (a program that produces its own source code as output). I've removed a few debugging print lines, unused functions, etc. to give you this cleaned-up version:
$ ./script.py testfile.txt
b = sys.argv
if len(b) > 0:
with open(b, 'r') as f:
filedata = f.read()
if __name__ == '__main__':
Explanation: I meant to have script.py grab the first argument to script.py, assume it was a file, and open and print it. However, I failed to actually check the behavior of
sys.argv ahead of time; turns out that the actual first item in
sys.argv is, in this case, "script.py", not "testfile.txt". You can try this out yourself, and verify that you'll get the same output whether or not you include testfile.txt as an argument. Off-by-one error. I should have had the
with open(b, 'r') bit try to
open(b, 'r') instead.
Reading a file is cheating in real quine competitions. But I still found this pretty funny.
# 15 Nov 2013, 03:44PM: Code4Lib, Open Data, Open Access, and Fighting Systemic Bias:
"Missing from Wikipedia" (code) makes me happy. I presented about it yesterday at Hacker School, asked a fellow HSer to discuss his critique of my code, and - live! on stage! - merged his pull request. Yay for code review and collaboration! (I also showed off a much sillier toy I made, which grabs some sentence from an English Wikipedia page if you give it a topic. Sample for "Chairs": "Some are decorative.")
I am grateful and proud that I can, with "Missing from Wikipedia," make a small contribution to the ecology of openly licensed code and content that I draw from. I could make "Missing from Wikipedia" because:
And so on. I fork from the repos of giants.
- the data for all Wikimedia projects is available under an open content license
- and queryable via an open-to-all API
- that lets you get information about 50 pages at a time (and with not-too-terrible rate limiting)
- that I could access using a good open source library with great docs
- available for an excellent and well-documented open source programming language
- that already Just Works with my source control system, text editor, operating system, and laptop
But we can only use a tool like "Missing from Wikipedia" if we have data to feed into it: a list of names. This is another way open data and open access to research is important. If we can get digital copies of things like the tables of contents of other encyclopedias and dictionaries, that makes it easier for us to systematically check for missing coverage on Wikipedia. But if those lists and tables are behind paywalls, then we can't see them.
And we need access to research papers, to help us figure out what tools to write. Let's say you'd like to fight systemic bias on Wikipedia and you want to write the most effective tool you can. What proportion of these citations on the effect of sexist language can you read & assess yourself? What proportion of the research that would help you do your job better is behind a paywall, and therefore not just hard to find, but essentially undiscoverable? Papers you can't link to are like missing Wikipedia articles -- out of sight, out of mind, out of the group discourse.
At this point I wave my hands excitedly and go off in some direction expounding on the intersection of open stuff (especially Wikimedia), social justice, comedy, and transformation. I presume I will cover similar topics in March 2014 when I keynote the Code4Lib conference, speaking to people who make things for/with cultural institutions. (Such an honor to be asked to keynote Code4Lib! And with Val Aurora of The Ada Initiative giving the other keynote!)
I've benefited so much from the ecology of open stuff. I aim to reciprocate, and to help make it even better.
# 13 Nov 2013, 08:38AM: Missing From Wikipedia: Tool to Help Fight Systemic Bias:
This week I wrote a tool I currently call "missing from Wikipedia" although the name may change. You feed it a list of people's names and the language Wikipedia you want to check, and it tells you who from that list does not currently have Wikipedia pages about them.
For instance, I gave it the ~2100 names from the table of contents from the Oxford Dictionary of African Biography (edited by Emmanuel K. Akyeampong and Henry Louis Gates), and asked about English Wikipedia. The list of people who (I think) do not have enwiki articles about them has 948 names. That means we do cover about half those Africans already, e.g., Nadine Gordimer. (This is an approximation, because I know some names need more finagling; for instance, currently the script messes up Barack Obama Sr.'s name so it wrongly thinks he doesn't have an enwiki page about him.)
I wrote this for Keilana (yay) as a tool to help fight systemic bias on Wikimedia projects. I hope other people find it useful. I've just added some code so that it prints out the percentage of missing people when it's done running, so you have a better measure of (for instance) French Wikipedia's coverage of important Senegalese leaders. I met Keilana in Berlin this past weekend at the Wikimedia Diversity Conference, and got to show her the power of APIs.
When I came to Hacker School, I had a general goal: "When I see a problem that could be solved by writing some Python and reading from/writing to an existing API, I want to recognize that and be able to solve the problem that way." Now I'm a little over halfway through and I have done it!
The code's GPL'd. Enjoy.
# (4) 06 Nov 2013, 08:06AM: Top, Iterators and Generators, and Git, Emacs, and REPL Tips:
Dumping into a post some things I've learned recently, trying to disregard the potential "you didn't know that already?!?!" surprise, feigned or genuine, that people might impose on me.*
* The magic of Hacker School: no one at Hacker School will do that. Nor well-actually me about this post! Random internet commenters might, and I may delete them.
- How did I never use top before? Magic! "Why in the world is my fan so loud? [run top] Epiphany, I closed that tab minutes ago, why are you still going like gangbusters? Fine, I'll quit and restart you."
- Lots of data types in Python are iterables. Like, say, lists, or strings. If you call the iter method with an object of that type as the argument, you get an iterator -- if you want to do stuff with that, then you give it a name. An iterator (holy crap) is like a function that holds onto state, so that it remembers what its state was the last time you accessed it! The point of an iterator is to traverse the iterable from beginning to end, yielding one value each time it's called with .next() or similar, then saying all done with a StopIteration error. Like this:
>>> a = [3,6,9]
[3, 6, 9]
<listiterator object at 0x7f5c8c8da490>
>>> s = iter(a)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
>>> r = "captain"
>>> w = iter(r)
- If the body of a Python function includes the verb yield (instead of return), then you've just made a generator. A generator creates an iterator that performs your whims! Again, you don't just call it directly; you assign a variable to a run of the generator function, with the same syntax as you'd use if you wanted to make an instance of a class, and then you have a generator object, which is an iterator that you treat as you would another iterator. Lemme show you:
>>> def foo():
... yield "first"
... yield "second"
... yield "last"
>>> b = foo()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
I read (bits of) so very very many pages about this, and fellow students tried to help me get this (thank you, Joe and Gideon), and then yesterday I paired with Jessica McKellar and she sealed the deal and I think I get it now! As she explained, you might want to use generators if you want to get an infinite series of values (e.g., all the even numbers up to infinity). Or if you're crunching numbers and it takes hella resources to do this particular part of the crunching on THE WHOLE DATASET ALL AT ONCE, with generators you can just crunch one input at a time and yield it up, then move to the next input in the sequence seamlessly when needed. You can speed up bottlenecks in your assembly line by doing particular computations in a just-in-time way.
- Meta-g g in Emacs takes me to a specific line number in the file. Putting (setq column-number-mode t) in ~/.emacs.d/init.el ensures that the statusbar at the bottom of the editor displays column number along with line number. These tips together make it much easier for me to seek out whatever discrepancies git or pep8 have brought to my attention.
- The xmlrpclib module makes it pretty easy to access XML-RPC web APIs, e.g. the Trac API as accessible on the Django project's site. However! The IPython and bpython REPLs may attempt to nicely autocomplete not-really-discoverable method names ... across the network ... and choke. And maybe crash. So if you want to play with it, just use the regular Python REPL. (But for everything else, oh wow, the bpython REPL is pretty snazzy.)
- git grep is great! It's automatically recursive, and only searches "the tracked files in the work tree, blobs registered in the index file, or blobs in given tree objects" (quoting from the man page). Just like with grep, if you use -n, then with every matching line you also get the line number. Or set lineNumber = True in the [grep] section of ~/.gitconfig to always have that on. If you miss colored output, use the --color=always option, or (as I just discovered) you should check out git configuration options, e.g. color.ui=true, to make LOTS OF OUTPUT colored and useful!
: Hacker School
# (4) 04 Nov 2013, 09:30AM: Comprehensions:
I spent a bunch of September in San Francisco, trying to tie up loose ends at work so I could go on my sabbatical with a free heart. My notebook says things like:
"30 is a large #" -- why? context
While there, I finally went shopping with Val and bought some new sneakers, so I could throw away my ratty old sneakers. I'd bought them in a fit of exercise-related optimism about seven years prior. I find it easier to buy clothes and shoes in other cities. I'm already off-kilter, disequilibrated, so why not add one more change, get one more bit of anxiety over with?
explain briefly when to use test 2 vs beta cluster
Say there will be 4 types of failures, then give numbers as you go
And during that trip, I went one step further: I went to a salon and got my hair dyed blue, like I'd wanted to for years. The dark blue only looks obvious in bright light, so people at work did double-takes, checking that their eyes' photoreceptors hadn't fritzed out. I'd never done anything that chemical to my hair before. I hadn't wanted to sadden my mom.
I got to Hacker School on September 30th and found out I was one of two women with blue hair. (We discovered quickly that we have a few mutual friends.)
The weather got cooler and cooler as we eased into our term and found our rhythms. The library got more books as people donated or lent them to the school; now there are huge gaps on the shelves as the books migrate to work tables. The kitchen has accumulated several different coffee-making gadgets, about ten containers of communal tea, and a steadily increasing stack of leftover paper napkins from takeout lunches. Most people sit in the same place every day now, as far as I can tell. Some prefer the beanbags, some the conference room with plenty of sunlight, some the standing desks, some the ABSOLUTELY NO TALKING quiet room, some the rooms with whiteboards, some the shared tables. I try to move around a lot.
For the first few weeks of Hacker School, I consciously basked in the number, diversity, and quality of the women in my batch. As the folks who run HS recently blogged, 42% of our batch of 59 are women. I look around the room and our chat channels and I see people helping and being helped, within and across genders. After the first week, I still hadn't learned all the women's names! Now I'm nearly used to the gender balance, but those first few weeks disoriented me in a good way, to tell the truth, and visiting non-HS physical and online spaces disorients me back. From the HS blog post:
One of the many benefits of having a gender-balanced environment is that, at least within the confines of Hacker School, the pressure to represent or focus on "women in programming" largely fades away, and people are free to focus on programming rather than rehashing tired arguments.
Focus on becoming better programmers: our guiding star. We try to avoid distraction (one guy said his phone battery lasts longer these days). But I feel guilt for enjoying our oasis and concentrating on myself, when I have so many sisters outside, wishing and working for environments a tenth as nurturing as Hacker School is.
But I have to focus on my own transformation right now, letting this experience change me, so I can go carry that transformation elsewhere.
I take a walk most days. I'd never spent much time in the Soho/TriBeCa region before, and now I'm getting used to the tiny blocks and the tourists shopping for knockoffs on Canal. The other day I saw, in my meandering, a shop window advertising "Maps and Dictionaries," which amused me, because I've been improving my fluency in Python maps and dictionaries, and generally grokking things like data structures and lambdas and whatnot.
It's heady stuff.
Yes, I like grabbing data from APIs and munging it, and I chortle when I can make the command line do new tricks. But oh wow, functional programming and hash tables make me clutch my head and shout superlatives and profanities. I'm beginning to get how mild-mannered programmers can turn into complete zealots about things like functional programming and structured data. Oh, who am I kidding -- I already thought I understood how people could do that, just for something to believe in, but now I see how I could turn into one of those evangelists, if this were the only revelation I'd ever had or thought I'd have.
My notes from the past five weeks include far less "tell $person about $thing" than usual:
Went to Python "office hours," learned stuff re setuptools & pip & virtualenv, and started Flask tutorial - got to Hello World, then step 2. Emacs improvements....
Stopped when angry/tired, wrote down summary, got beer, got Joe, figured out was editing file that was not getting run (venv), started getting stuck in dependency hell (mysql?!) when checking whether problem was BZ-specific. Stopped for the day....
Some transformations make us over all at once, the same function applied uniformly to every element in a collection, from black hair to blue in an afternoon. Some happen to parts of us first, before other parts catch up, eventually consistent. I'd been programming for a long, long time before I called myself a programmer. I can't tell whether I feel arrived yet, whether I feel home. (We talk about progression in time as though it is progression in space, don't we? As though our lives are journeys, as though our schoolteachers are packing our saddlebags, as though a calendar is a map of time.)
Last week, Leonard and Beth made brownies with marshmallows and M&Ms. I taught a few peers at Hacker School to play Once Upon A Time. Leonard and I watched "Wives", a feminist Norwegian seventies film. I learned lots of little things about zip, map, filter, reduce, databases, packaging, bpython, bash. I dressed up as "Futuristic Businesswoman Sumana" for Hallowe'en, in my green business suit that looks vaguely Vulcan (lapels are illogical). I got to question 11 in Python Challenge. I'm in the middle of reading about eight books. The dead leaves started piling up on the sidewalk, fun to crunch through, and the autumn rain started, although Saturday the sun stayed out. I walked to the theater and thought, it won't be this warm again for five months.
Every few days I remember that Aaron is still dead. And I think I dreamt about my dad a few times in October; in one dream I got confused, thinking, "wait, I thought he died already, how could he be dying again?" but that's something you don't say to the rest of your family, or at least something I don't say. I think I've gotten to the long prairie of life where I'll be going to more funerals than weddings from here on out.
In September, in San Francisco, a colleague asked me: why all these changes all of a sudden? The sabbatical, the hair, the shoes? And I asked whether she remembered Aaron Swartz. She hadn't known him, but she remembered the public mourning of his death. I told her what he'd said, the revolution will be A/B tested, and explained what he'd meant. We activists have a responsibility to use our energy well. I, in particular, believe I need to become a better software engineer so I can be a better social engineer. So, I told her, I drew two relevant lessons from Aaron's death:
- Life is short, so be a better activist.
- Life is short, so do small harmless things that make you happy.
Today I'll put on those new shoes and go to Hacker School, and drink tea, and learn from women and men some new thing that makes me swear aloud, that will help me fight. Everything that lives changes; the only way to stop changing is to die. If I find myself afraid of growing, I'll remember all the forces that don't want me to learn. Death being only one of them.
: Hacker School
# 02 Nov 2013, 02:25PM: Emboss:
I recently came across Lauren Bacon's "The Accidental Boss: Making Peace with Power" again, and it reminded me: We don't talk enough about power. We don't talk enough about how hard it is to transition from individual contributor to manager, and to delegate the tasks that you really love, that might even constitute your identity. We talk about delegating, but we don't talk enough about the inner emotional security you need to develop in order to hire and trust people smarter than you.
And we certainly don't talk enough about the necessary skill of constructively managing your anger in the workplace.
We say that anger is poison or that anger is righteousness, but have you had a role model who showed you how to manage your anger? Have you learned when to wait before sending that pissed-off email? How did you learn that?
And those intersect, of course. Sometimes I disagree with my subordinates or my superiors, but I believe I always work with them constructively and I don't let my mood get in the way of hashing out the issues and finding a decision. But what if I'm wrong?
Argh gender. We women get disproportionately less training, formal and informal, in handling personal power and in using anger. And I have to do that double-checking multiple times a week, predicting how others would react to any given reveal of my power or anger.
Jono Bacon publicizes the risk of burnout. Those middle stages include substantial anger, irritability, and anxiety. How do you know when your anger is a healthy, legitimate response to a wrong? How do you know when your anger is getting in your way?
(Oh, and those of us who grew up with parents who didn't deal with their own anger responsibly have even more trouble with this. Double argh.)
What do we have? Where are we talking about these things? Sunday sermons, "Chain of Command" and "Lower Decks" from Star Trek: The Next Generation, the odd thoughtful BDSM-related blog post or fanfic, a few essays about Obama's leadership style, leadership coaching seminars, activist retreats? Is this what the Harvard Business Review is for?
I have gotten into the habit of reviewing my anger with a trusted colleague or friend. "Foo happened and bar happened and he said x and I said y... I feel frustrated/resentful/unappreciated/patronized, and basically angry, and it's distracting me... what do you think? am I being reasonable?" Advantages: fewer damaging blowups. Disadvantages: sometimes I lose the opportunity to respond to a problem in the moment, and when I do respond, the other person thinks I'm holding a grudge.
Skill acquisition is hard, yo.
# 01 Nov 2013, 11:06AM: PEP 8 Compliance:
It's easier to read and contribute to code when it's stylistically consistent. This is a reason why we have PEP 8, the style guide for Python code. It says things like:
Avoid extraneous whitespace in the following situations:...
Certainly most of the Python code I run across follows that convention. So I got confused when I read Bicho code that sometimes had extraneous whitespace between function name and arguments. Sometimes it did and sometimes it didn't. From a note by one of the maintainers I inferred that Bicho's developers want code to comply with PEP 8.
Immediately before the open parenthesis that starts the argument list of a function call:
No: spam (1)
So I decided to look for those discrepancies, so I could fix them. You can use the
pep8 module to find instances of PEP 8 noncompliance, and you can give it arguments to narrow down to just one issue. A command like
$ python pep8.py --select=E211 Bicho/
gave me the list of lines with extraneous whitespace before the open parenthesis. (I've edited out some path-related cruft.) I thought I'd write a regex to fix those lines, but Julia and Leah kindly talked me into seeking out a pre-existing tool first, and I found Pep8ify.
$ pep8ify -f whitespace_before_parameters Bicho/
gave me the proposed fixes as readymade diffs. To make Pep8ify do those fixes:
$ pep8ify -f whitespace_before_parameters Bicho/ -w Bicho/
So now I've filed an issue with a pull request. (I also used Pep8ify to clean up some whitespace inconsistencies around operators like "+" and "=" while I was at it.)
Thanks to Szymon Guz's blog post for pointing me in the right directions.
: Hacker School
# (1) 28 Oct 2013, 11:59PM: On Ability:
Someone discovered "that the addition of 'Harry' to almost any Plato quote makes it seem legitimately like a nugget of wisdom out of the mouth of Albus Dumbledore." This reminded me to look up my favorite Dumbledore quote:
It is our choices, Harry, that show what we truly are, far more than our abilities.
I am trying to remember that, because every day I go to Hacker School and sit next to people with lots more programming skill than me, and sometimes I find that discouraging. Or I realize how badly I want to impress people, to feel admired and respected, and how that sometimes gets in the way of growing and achieving actually admirable, respect-worthy things. I need to remember to disregard that kind of anxiety fungus emotion. Thomas Beagle said in some related comments:
to be a good geek you [have] to have both humility and arrogance in equal measures. The humility was so you'd admit you didn't know something and get help/read the docs/etc., the arrogance was the bit that said "I don't know that now... but I can and I will soon."
I think that, like a lot of people, I conflate skill and confidence, and I need to disassemble a construct I didn't even realize I had in my mental infrastructure. How slippery, that the confidence I need to develop is the confidence to express uncertainty, to say "I don't understand" as many times as it takes. Our Hacker School facilitators guide us to try projects that intimidate and scare us. Truly being vulnerable to my own ignorance is on that list. I wish I knew how to credibly and persistently promise myself that the rewards from being open to change are greater than the return on inertia.
: Hacker School
# (1) 28 Oct 2013, 02:20PM: Prank And/Or Business Idea:
Problem 1: Food at tech meetups usually fails to satisfy foodies. Make-your-own-tacos if you're lucky, "the Statue of Liberty is crying" bad pizza if you're not.
Problem 2: Most places we hold evening tech talks just aren't very good for actually hearing and enjoying those talks. Either the sightlines and sound suck, or you're in some weird institution like a college or Google that comes with a bunch of other emotional freight.
Problem 3: People want entertainment with their dinner, gimmicky outings for their friend or professional groups, and opportunities for limited socializing with strangers.
Proposed Solution: a proper restaurant like those old 1940s halls, the ones that used to show up in set pieces in Astaire movies. The terraces of round tables, where you might be seated with a few folks you don't know yet, remain the same. But the floor show is a few short tech talks, nightly.
# (1) 26 Oct 2013, 05:07PM: Some Artifacts:
At Hacker School, I'm working on little projects to teach myself various things. I am following my own advice by embracing silliness. A few things I have made, all of which now have code up on GitHub:
An Obama speech generator. I wrote this command-line speech generator just after Barack Obama gave a televised speech partially about trouble with HealthCare.gov. I thought, "what if Barack Obama gave LOTS of speeches about tech?"
Also, I wanted to try out test-driven development, so speech-tests.py has the tests I wrote (using Python's unittest module) before or as I wrote functionality.
So, here you go. Run speech.py at a command line and type in three tech buzzwords when asked. (Alphabetical characters and spaces work, but no other punctuation -- "the cloud" and "NoSQL" are fine, but "object-oriented" won't work.) You'll then get a short speech incorporating at least one of the buzzwords you've provided. This gets Leonard to laugh a lot at lines like "Thank you, and may God bless Agile."
"Personality Rights", a super-short and moody game. I made it in Ren'Py, a platform for making "visual novel"-type games, which meant I fiddled with a sort of domain-specific subset of Python to specify plot, characters, styling, music, and graphics. (Graphics include the turtle image at right.) I wrote it in three hours because that's the time limit for the Ectocomp game competition.
A scifi novel title generator (previously).
Improvements to Bicho, although I backed off my giant "refactor and add tests to everything" plan in favor of learning to write tests first.
(I wish I were using Gitorious instead of the closed-source GitHub, but Gitorious wouldn't let me log in, even after password reset. Bleah!)
What have I learned? I only have eight weeks left. I'm better than I was at Python, emacs, git, and bash. I have gotten to question ten on Python Challenge, and finished all of CodingBat Python.
Sometimes I have a reaction I don't like. When I learn some new amazing thing you can do with Python, I get angry that I didn't know it already. It's a very fixed-model way to react, not a growth-model way, and I think I have to fix more things in me before that stops happening.
: Hacker School
# (1) 26 Oct 2013, 12:42PM: Hacking Music:
I spent a day at Hacker School without a working pair of headphones recently. So I spent about three hours pairing, which I loved and which I'm going to repeat every day that I can. But I also booked a room to play music in and invited fellow participants to come in and code while listening to music together for an hour. I was just going to play some video game soundtracks off my Nokia N9, but Andrew showed up with his massively multiplayer music player and the speakers we use for parties. I know when I'm outclassed.
When I ask my colleagues what they like to hear while coding, we mostly agree: "nothing with words I can understand." So, here, monolingual English speakers have a big advantage! Some albums I like:
And then there are the songs I listen to on the subway when I am discouraged from bughunting and troubleshooting. These are my "Perseverance!" songs.
- Greatest Marches (various composers and performers). The original techno music!
- Beirut, The Gulag Orkestar
- Eric Skiff, Resistor Anthems
- Wendy Carlos, Tron soundtrack (original Tron)
- Daft Punk, Tron:Legacy soundtrack (like everyone else, I also enjoy the "Reconfigured" remix album)
- El Ultimo Skalon, Ciudadano del mundo
- Grand Valley State University New Music Ensemble, In C Remixed
- "Tubthumping" (They Might Be Giants version)
- "Give Paris One More Chance" by Jonathan Richman
- "Be Born" by Tally Hall
- "The Sadder but Wiser Girl" from The Music Man soundtrack
- "This Too Shall Pass" by OK Go
- "Smells Like Teen Spirit" by Nirvana
- "Get Around" by Leonard Richardson
- "Better Times are Coming" by Kate and Anna McGarrigle, from "Songs of the Civil War"
- "You've Got To Do It" (lyrics and tune by Mr. Fred Rogers, interpretation by Holly Yarbrough in Mr. Rogers Swings!). Some lyrics I have listened to over and over:
If you want to ride a bicycle and ride it straight and tall
You can't just sit and look at it, 'cause it won't move at all
It's you who has to try it
And it's you who has to fall
If you want to ride a bicycle and ride it straight and tall
: Hacker School
# 22 Oct 2013, 08:25AM: OMG, or, Biting Off More Than I Can Chew:
So, there are random non-programming reasons why I didn't feel like I made much progress yesterday -- I tried working in a beanbag chair (no good for Sumana), I drank coffee too early in the day (caffeine crash in the afternoon) and I listened to the wrong music (Guster and Neutral Milk Hotel? might as well be The Mountain Goats for all the good that does me) -- but here are the big ones.
Today is a new day! As soon as I hit Publish I will pair with Leonard a little. Yay morning freshness.
- When I got frustrated, I didn't get help as much and as often as I should have. This is DESPITE the fact that I DID get help from Zach, Allison, Stew, and, crucially, Mary. I had short conversations with the first three, and then Mary paired with me for 45 minutes and helped me understand some things about my goal of refactoring a big codebase. Such as "it is a big job" and "you should have a guiding thing you're trying to do as you go, like adding a new feature, or adding tests". Also she helped me see how to write a test with little mocks so that I'm not doing the foolish thing of testing "does the exact same technique (as the function I'm testing) work on the exact same input (as the function I'm testing is using) and get me the same output (ibid.)?"
- When I was trying to play with Beautiful Soup for the first time, I used a test page different from the test page in the documentation/tutorial. Then, when I ran into seemingly inexplicable errors, I didn't think to look at the differences between my input data and the input data in the documentation examples. I felt especially helpless because my spouse wrote Beautiful Soup and thus it seemed like I ought to wait till I got home and ask him for help. But that broke my momentum; I should have swallowed my embarrassment and pride, and asked someone at HS for help.
- To feel less like a broken and incompetent programmer, I did some simple Codecademy exercises, but the interface feels slow and -- since I already know the concepts in the exercises I was doing -- I knew I wasn't really learning anything. I should have skipped them, or just skipped to something hard.
- I paired with Julia to understand some gzip stuff to hopefully help her debug a problem, which means I learned some things about Huffman coding and the mind-breaking way that LZ77 encoding and decoding works. But it took what felt like a super long time because I never learned some basic CS things and I am not facile with binary arithmetic, and I felt like a drag, and felt blergh. Maybe I was just in a down mood; maybe I should have just bid Julia goodbye and gone for a walk or something. (Sorry for my raincloud, Julia!)
- We heard a talk (in our Monday night talk series) that somewhat went over my head, and which, as I realized about 25 minutes through, was simply going to be hard for someone with my learning style. I an an active enough learner that hearing about concepts like event loops and threads (and the problems concomitant with those approaches to concurrency) isn't enough; I need to play with them and experience them. I am a visual enough learner that, if I haven't tried writing concurrent programs before, I need diagrams or animations or similar visual elements to help me understand what works and doesn't work, rather than just sentences spoken orally. I am a sequential enough learner that if I don't get the first concepts in a presentation, it's going to be hard for me to grok any of the middle. And I am a sensing enough learner that I really need to understand the examples, and I had a hard time reading the syntax of an unfamiliar language (OCaml) to get at what was happening in the examples. (This is interesting data because I thought I was way more on the intuitive side of the sensing-intuitive spectrum, and the verbal side of the visual-verbal spectrum, and the reflective side of the active-reflective spectrum. Failures show us nuance!) So, for those first 25 minutes, I mostly felt unintelligent, and tried to follow, and felt my morale sag. After I realized "oh, this is almost exactly the opposite of my learning style", I felt less bad. So in the future I shall try to have that realization earlier. (Monday night talks are lids-down, so it's bad manners to try to understand the speaker by writing code. Even though I consider myself a crap artist, maybe I should try getting out pen and notebook to draw my own diagrams in cases like this.)
- After I came home, Leonard worked with me on Beautiful Soup play and helped me understand another perspective: writing tests is writing code, and writing tests is hard -- in fact, generally harder than writing the code that the tests test. You have to think on another plane. Oh. I've been blithely walking into intensely difficult non-solved-problem areas of software engineering, viz., refactoring and testing, with a codebase entirely new to me, as a not-very-experienced programmer. No wonder I've been running into difficulty.
: Hacker School
# 18 Oct 2013, 11:17PM: A Mediocre Day But A Good Week:
Mel Chua visited Hacker School last week and especially entreatied us to blog about bad days, days we felt demoralized or unproductive. It helps her with her research. Well, Mel, here you go.
It's a Friday, and we get Fridays "off", that is, we don't have daily checkins and the facilitators don't have bookable office hours and some of my colleagues are gone and it feels muted and off-kilter. On September 27th I'd thought that I'd be working four days a week, but I look at colleagues who come in every single day, including weekends, and feel FOMO. Fear Of Missing Out. So I've started coming in for half a day on Fridays, but it rarely feels as good and productive. I think by the end of Thursday I could really use a break to recharge. So that's one thing. I think from now on, if I come in on Fridays, it should be to accomplish a very specific task, and I should leave when that task is done or when it becomes clear that my energy or cognition is flagging.
Last week some of us joked around that we should do the opposite of Casual Friday: Fancy Friday we dubbed it. So today I came in wearing a dress, a bit low-cut at that. But I didn't see anyone else in suits or gowns or similar, so I didn't feel as comfortable in what I was wearing. For next week, if I come in on Friday, I may try a pantsuit.
I arrived around lunch time I and brought my lunch, which is good because it makes me feel good to be frugal. But I read a depressing and not particularly edifying message board as I ate, and I should probably save that kind of thing for the weekends or my couch at home.
I started off my work with the vague goal of "learn about unit testing" and it took me a lot more reading, sighing, and moping time than it should have for me to ask for help (thank you, Ryan, for our impromptu chat in the kitchen that led me to understand when to use assertions and when to go for mocks, and thank you to HSers who chatted with me about Mock) and to reduce my aims to something more manageable. Next time: follow my own advice, and ask for help after fifteen minutes of feeling stuck. Also, "learn about x" is an okay way to start surveying the problem space and the solution space, but "try a single implementation/example of x in a toy app" is a much better goal for an afternoon.
I drank coffee when I should have had water. I ate licorice when I should have snacked on edamame. So I got jittery and sugar-crashy instead of focused.
I chatted with someone, and I was judgy or negative when I could have been more thoughtful and constructive.
I helped people with git problems and questions, which I'm glad about, but I missed the opportunity to ask them about their learning styles first and organize my thoughts a little accordingly. It felt haphazard.
It took me way too long to start listening to energetic productivity-provoking music on my music player; maybe I'll just set a reminder to make that happen around 11am every weekday.
And there's random other stuff on my mind, e.g., having to grab my old mail off the OCF's servers by Sunday.
So, Mel, overall, today I started off in a low-energy, non-driven mood, and I didn't take the kinds of steps I know I oughta take in order to fix it. But nearly every time I spoke with someone, it shook me out of my rut, and helped me gather the activation energy to do The Right Next Thing. So it could be that Fridays I should just try to arrive in the morning and set up a pact with a few colleagues to do a check-in conversation every 30 minutes. (It's easier to set up that sort of thing upon shared arrival in the morning.)
Hope that helps you. It helped me.
What did I accomplish today? I implemented a few docstrings and started learning how to use __repr__. I showed some people how to work with branches and multiple remotes in git, and how to fork and make pull requests on GitHub. I reported a few bugs in one product and made a pull request for another. I used "git cherry-pick" for the first time, with Alan's guidance. I wrote most of a test that uses "assert" to check that there's a path from Independence to Portland given the links between cities in my game. I got emacs to give me two side-by-side buffers. 3.7 hours tracked in Project Hamster, 2.9 of learning and .8 of teaching, plus a few more of faffing about on the net or in conversation. But I surpassed my 20-hour learning goal for the week -- I'm around 25 -- so this week overall I've done well.
In retrospect, today felt suboptimal in contrast to a usual Hacker School day. Today I plateaued. I think after some rest this weekend I'll plunge in fresh on Monday with clear goals and better discipline.
: Hacker School
# 16 Oct 2013, 11:19PM: Idiosyncratic Troubleshooting Tips:
Yesterday I tried to diagnose and fix a bug in an open source project. I got discouraged because of a few factors, so I'm noting down a few things I ran into, for future Sumana and other similar folks.
Thanks to Joe, Fei, Rupa, Allison, Travis, Moshe, Kat, Julia, and the Bicho developers for their help the past few days!
- Are you editing the right file? If you're in a virtual environment, make extra special sure that the file you're trying to tweak is the same one listed in the traceback.
- Special characters? For instance, bash might get all weird on you if there's an ampersand (&) in an argument you're passing via the command line.
- pdb, assertions, and IPython. I'm working in Python, and I've started to learn to use "python -i", the Python debugger, "assert" tests, and the IPython toolkit. IPython especially is cool because the visual presentation of the stack trace is easier to follow.
- The database setup toolchain is blergh but worth it. If a project needs a MySQL database set up, then fine. There's a little bit of dependency hell but it's not intractable, especially if you have someone nearby who's done it before. What all did I have to do? I can't retrace the order, but from looking at my .bash_history, Synaptic history, and dpkg.log:
And then, you know, you have to do the initial privilege-setting and connection-making, and probably create a database, blah blah blah. But! Eventually it works and you can alter and create and drop things like it's going out of style. (Which it probably is, memory bank fashion going the way it is.) And it does eventually work, and the stack of dependencies doesn't REALLY take up loads of disk space the way it feels like it will.
- apt-get install python-dev
- apt-get install python-mysqldb
- apt-get install mysql-server
- apt-get install libmysqlclient-dev
- apt-get install mysql-common
- pip install MySQL-python
- pip install python-MySQLdb (I think?)
- Branch! Not quite as relevant, but: just get into the habit of proper git hygiene when working on improving a shared codebase, e.g., switch to a new branch for a new logical set of changes. It makes merge requests/pull requests so much more frictionless. And then "git checkout -" makes it super easy to switch between the branch you're on and the last branch you were working on.
: Hacker School
# 09 Oct 2013, 10:43PM: Programming Jokes:
The Hacker School application form asks you to provide some code you've written so the faculty can look at it. I wrote a game: "Where on the Oregon Trail is Carmen Sandiego?" It is a joke of a game and a platform for further jokes. During my first week at Hacker School, I improved my programming skills by improving it. For instance, now multiple villains might have stolen that wagon tongue, including Waldo.
Kat Walsh encouraged me to actually implement the joke I made in August. So I am now working on a toy web app (using Flask) to grab physics article titles from English Wikipedia (via the MediaWiki API, via Pywikibot) and perform Queneau assembly on them to make plausible scifi novel titles, and then display those strings on a web page. So far, fun titles have included:
They make people laugh. With software, I can scale my comedy! I can make more people laugh at more things. I think we could get more people programming if we showed comedians that you can pull better pranks if you can code.
- Optical Reluctance
- Hazard Steel
- Electrodynamic Hackerman
- Joule 1584
- Choke River
- Nernst Hopping
- Source Cloaking
- Joule Summation
- Waveguide Bearing
- Capacitance Torus
- Ionic Agent
- Tunnel Curve
# 06 Oct 2013, 12:00PM: What They Don't Know:
Or: you are an expert if you can save people time.
Late in 2011, I found out that one of my colleagues, a whip-smart and infinitely organized administrator, wanted to know more about how the engineering side of Wikimedia works. So I started teaching her. Every month, we talked for about an hour. She asked me about some activity from the monthly report and I explained what we're doing and why, often using analogies. She loved it and felt far more connected to what her other colleagues were doing.
She's not at Wikimedia anymore, so I have tried doing it as a Wikimania presentation and continuing the tradition with other WMFers who were interested. So far I've done a lot of one-off "What the fudge does Wikimedia engineering do" sessions for incoming folks, mostly non-engineers coming into the Foundation's other departments.
Two lessons from that experience:
- Sure, continuing mentorship relationships are awesome. But don't discount the value of a few limited teaching sessions.
- I have about three approaches to teaching this stuff: Historical (What has happened since we started in 2001?), Experiential (What happens under the hood when you go to en.wikipedia.org in your browser, and who's in charge of what parts?), and Organizational (Who are the eight directorates in WMF engineering, and who are other important Wikimedia tech institutions, and who does what?). I want to get better at the historical mode, which means learning what happened in what order between 2001 and 2011; right now I do the org-chart mode quite well, and the experiential mode well except for talking about load-balancing and caching.
I wish I'd kept good notes of all the questions people have asked during these sessions. Some of them:
- What is a parser?
- What is LAMP?
- What is MySQL? What is a database?
- What is Apache?
- What does "open source" really mean?
- How can it be that so many talented programmers are only in their twenties?
- What is the role of the Engineering Community Team?
- What do the people in the MediaWiki core team do?
- What is Subversion, what is Git, and why did we switch?
- Do all the Wikimedia sites run on MediaWiki?
- How can we do what we do with so little staff?
- What's with this Lua thing?
- Why has it taken so long to write the Visual Editor? (This question led me to sketch out a blog post we published.)
- What is a "virtualized hosted development environment" (Labs)?
- Why did we have to switch to IPv6 and why was that hard?
- What is an API?
- What is HipHop and why would we use it?
- Why did we work on a specialized Wiki Loves Monuments app?
- What are the Universal Language Selector and Milkshake?
- What is Swift?
- What is the difference between the E2 (Editor Engagement) and E3 (Editor Engagement Experiments) teams, and what do they do? (We partially fixed this by rearranging and renaming the teams to Core and Growth.)
- What is HTTPS? How does SSL work?
- What kind of security problems could a web-based application have? Do we have worse problems because we're open source?
- What does a product manager do?
- Why don't we provide automatic translation from and to different language Wikipedias?
- What is Wikidata?
- Is Wikipedia Zero just for Wikipedia, or also the sibling sites?
- How do we consult with hundreds of different wiki communities when building and rolling out our software, especially when we don't speak their language?
I have just started at Hacker School, a place designed to help everyone learn. That means making people feel comfortable with saying "I don't know". I've benefited countless times from this, because if no one's going to belittle me for not knowing something, I feel safer asking and learning. I didn't realize how much I would also get to teach! When everyone feels safe saying "What does that mean?" then I get to help more people learn more things. I've explained, among other things:
It's super amazing when you teach someone a skill or a perspective that changes them. I feel so lucky that I am an expert, i.e., someone who can save other people time. It is a form of hospitality.
- what Markdown is, and why you would use it
- what screen-scraping is, and why APIs would be better
- how I use git
- what unit tests are, and why you would add automated testing to your project
- dozens of opportunities to reuse, integrate with, or improve Wikimedia data and software
- a bunch of Unix command-line tips, such as control-R for interactive search of bash history
- the sordid history of ReiserFS
- why Nvidia drivers are the classic example of "proprietary stuff that's not in the Linux kernel but that you might want to use so some distributions carry it"
# 06 Oct 2013, 10:41AM: Net Work:
I saw Apollo 13 when it came out in theaters in 1995, and I don't think I'd seen it again until the other night, when Leonard and I rewatched it.
Leonard and I both thought it was pretty inspiring -- see his review -- and of course we both absolutely loved the Iron Chef moment where engineers on the ground gather copies of all the equipment the astronauts have, dump the mess on a table, and get to work kludging together a CO2-scrubbing solution.
What is a hero? Is it a person who makes a big positive delta between the world as it might have been and the world as it is with the hero in it? (I'm remembering when Aaron Swartz talked that way, when I asked him what he wanted.) Part of the magic of that scene in Apollo 13 is that it demonstrates that you can be a hero in a life-or-death situation even if you are not the one at risk. John Rogers's recollection from filming a Global Frequency pilot touches on some of the same chords, about mutualism and solidarity and sharing our brilliance with each other towards an urgent goal.
My tastes in stories grow clearer over time. I like wit, compassionate clear-eyed observation about how intelligent people act, and celebrations of labor. Via deepad: "Yeh Zindagi Bhi" from Luck By Chance.
Part of the joy of open source is that I can work with and share my work with more people than I could otherwise. I can help people I've never met. And I get help from people I've never met. Some days you're Mission Control, and some days you're the astronaut.