Cogito, Ergo Sumana

Categories: sumana | Open Source and Free Culture:Transparency in Government Code

Algorithms and software governments make and use should be available for public inspection and reuse


: Supporting Int 1696-2017 for Source Code Transparency in New York City: The principle at stake in California v. Johnson: due process requires that we be able to examine the evidence used to convict someone. Kern County got a $200,000+ grant and started using closed-source software to perform a new kind of DNA testing for criminal forensics. You are not allowed to audit the software to check for bugs, but the company founder will fly in and testify in court to say he attests to the validity of the results it finds. Uh, no, we need to check, and the ACLU and EFF have just filed amici curiae* briefs before California's Court of Appeal for the Fifth District, saying so.

Man at lectern in front of screen displaying 'Winning Raffle Numbers: 12345 12345 12345 12345', photo (used by permission) by Mike Pirnat at the PyCon PyLadies auction in 2017As I've written and even testified, we need more auditability, transparency, and security in software governments use in laboratories and field tests. Heck, we need it in software governments use to make decisions more generally -- lotteries for visas, school assignments, parole and prison sentencing, and so on.

So I was delighted to learn of bill Int 1696-2017, currently before New York City's City Council. Summary:

This bill would require agencies that use algorithms or other automated processing methods that target services, impose penalties, or police persons to publish the source code used for such processing. It would also require agencies to accept user-submitted data sets that can be processed by the agencies' algorithms and provide the outputs to the user.

I applaud James Vacca, chair of the council's Committee on Technology, for introducing and sponsoring this bill, and for citing/shouting out to danah boyd, Kate Crawford, and Cathy O'Neil as people whose work has shaped this legislation. The New York Times says: "As a committee chairman, he plans to convene hearings before he leaves office in December." I'm looking forward to attending those hearings.

If you live in New York City, you can contact your councilmember and suggest they cosponsor this bill. If you live elsewhere, consider telling your local elected officials that they oughta introduce legislation like this. When writing or calling, if you're a programmer or other technology expert, say so -- our voice matters.

I have more links in the algorithmictransparency tag on Pinboard.


* Many years ago, Seth Schoen made me an illustration that we still have somewhere. Reconstructed from memory:

[one smiling stick figure, male, near a courthouse] Sum amicus curiae.
[one smiling stick figure, female, near a courthouse] Sum amica curiae.
[many smiling stick figures of various genders, near a courthouse] Sumus amici curiae.
[one stick figure, male, holding a finger to his mouth as though shushing you, near a courthouse] Tacit! Sum inimicus curiae!

Edited Tuesday Sept. 19th to add: The Committee on Technology is holding a public hearing to discuss Int 1696-2017 on Monday, October 16th.

Filed under:


(2) : Transparency And Accountability In Government Forensic Science: Sumana Harihareswara next to public hearing noticeIn February, I learned that the New York State Assembly was planning a public hearing on government oversight of forensic science laboratories, and then was invited to offer ten minutes of testimony and then answer legislators' questions. This was a hearing held jointly by the Assembly Standing Committees on Codes, on Judiciary, and on Oversight, Analysis and Investigation and it was my first time speaking in this sort of capacity. I spoke on the importance of auditability and transparency in software used in devices the government uses in laboratories and field tests, and open source as an approach to improve these. And I testified to the efficiency, cost savings, security, and quality gains available by using open source software and by reusing and sharing open source software with other state governments. Here's a PDF of my testimony as written, and video and audio recordings are available as is a transcript that includes answers to the legislators' questions. It is a thrilling feeling to see my own words in a government hearing transcript, in that typeface and with those line numbers!

As I was researching my testimony, I got a lot of help from friends who introduced me to people who work in forensics or in this corner of the law. And I found an article by lawyer Rebecca Wexler on the danger of closed-source, unauditable code used in forensic science in the criminal justice system, and got the committee to also invite her to testify. Her testimony's also available in the recordings and transcript I link to above. And today she has a New York Times piece, "How Computers Are Harming Criminal Justice", which includes specific prescriptions:

Defense advocacy is a keystone of due process, not a business competition. And defense attorneys are officers of the court, not would-be thieves. In civil cases, trade secrets are often disclosed to opposing parties subject to a protective order. The same solution should work for those defending life or liberty.

The Supreme Court is currently considering hearing a case, Wisconsin v. Loomis, that raises similar issues. If it hears the case, the court will have the opportunity to rule on whether it violates due process to sentence someone based on a risk-assessment instrument whose workings are protected as a trade secret. If the court declines the case or rules that this is constitutional, legislatures should step in and pass laws limiting trade-secret safeguards in criminal proceedings to a protective order and nothing more.

I'll add here something I said during the questions-and-answers with the legislators:

And talking about the need for source code review here, I'm going to speak here as a programmer and a manager. Every piece of software that's ever been written that's longer than just a couple of lines long, that actually does anything substantive, has bugs. It has defects. And if you want to write code that doesn't have defects or if you want to at least have an understanding of what the defects are so that you can manage them, so that you can oversight them (the same way that we have a system of democracy, right, of course there's going to be problems, but we have mechanisms of oversight) -- If in a system that's going to have defects, if we don't have any oversight, if we have no transparency into what those instructions are doing and to what the recipe is, not only are we guaranteed to have bugs; we're guaranteed to have bugs that are harder to track down. And given what we've heard earlier about the fact that it's very likely that in some of these cases there will be discriminatory impacts, I think it's even more important; this isn't just going to be random.

I'll give you an example. HP, the computer manufacturer, they made a web camera, a camera built into a computer or a laptop that was supposed to automatically detect when there was a face. It didn't see black people's faces because they hadn't been tested on people with darker skin tones. Now at least that was somewhat easy to detect once it actually got out into the marketplace and HP had to absorb some laughter. But nobody's life was at stake, right?

When you're doing forensic work, of course in a state the size of New York State, edge cases, things that'll only happen under this combination of combination of conditions are going to happen every Tuesday, aren't they? And the way that the new generation of probabilistic DNA genotyping and other more complex bits of software work, it's not just: Okay, now much of fluid X is in sample Y? It's running a zillion different simulations based on different ideas of how the world could be. Maybe you've heard like the butterfly effect? If one little thing is off, you know, we might get a hurricane.

Filed under:


: Upcoming Talks: I happened upon the New York state Assembly's website last week,* and noticed an upcoming hearing about "Government oversight of forensic science laboratories" (PDF), hearing oral testimony by invitation only. I wondered: Who's on the list of witnesses? And will any of them talk about the danger of closed-source, unauditable code used in forensic science in the criminal justice system?

I followed up, and we got me, plus Rebecca Wexler, the author of that piece, invited to speak. We're testifying tomorrow, Wednesday, February 8th, in New York City. In preparation, I'm conferring with Karen Sandler of Software Freedom Conservancy (who was slotted to speak but now can't) and with acquaintances who work in government forensic labs.

I did speech and debate in high school so in some sense I have been preparing for this for twenty years.

A little further off:

Next week, I will participate in the WONTFIX Cabal (Maintainerati) unconference for open source maintainers on February 15, 2017, in San Francisco, California, USA.

I will give the closing keynote address at LibrePlanet, a free software conference, March 25-26, 2017, in Cambridge, Massachusetts, USA. Tentative title: "Lessons, Myths, and Lenses: What I Wish I'd Known in 1998."

I will be one of the Guests of Honor at Penguicon, an open source and science fiction convention, April 28-30 2017, in Southfield, Michigan, USA.


* via Lauren Sperber's blog post about "the New York State Reproductive Health Act to get abortion removed from New York State's Penal Code"

Filed under:


: Entertainment Benefits of IDNYC Card: The new IDNYC card is free, government-issued photo ID for New York City residents. "Immigration status does not matter." That is to say, people who are came to NYC from abroad, and currently don't have legal documentation to support that, can get this card. Which is great -- it gives everyone, including them, a way to start banking, get access to schools and have something to show to hospital receptionists. It also works as a library card, and has a bunch of other benefits. Also, the application's gender options are:

Friends of mine are getting their cards for the free memberships at the New York Botanical Garden, American Museum of Natural History, MoMA, Museum of the Moving Image, and dozens of other museums.

I was curious about the entertainment benefits, specifically, cheaper movie and theater tickets ("Movie Tickets as low as $8.00"). In order to get those benefits, you have to register at MemberDeals.com, a for-profit website run by Entertainment Benefits Group, Inc. And the site does not give you specifics about what you can expect if you register; you have to register in order to browse deals. The IDNYC site is pretty specific about the other benefits, and I'd like to know more before I register. So, in another installment of "I make phone calls to closemouthed organizations and then blog the results", I phoned up their customer service line.

I think the privacy policy strongly implies but doesn't state that EBG keeps a record of the purchases you make; the customer service rep I spoke with specifically said that EBG does not hold onto your credit card number if you make a purchase. (Which is important for PCI compliance, of course.) It seems unclear to me whether they keep a record of the discounted tickets users buy through them.

Registered members can expect special offers emails about biweekly, and can always unsubscribe.

The customer service rep did not give any examples of specific amounts in current discounts EBG offers its members, e.g., "$50 for such-and-such a ski ticket." But she said that the EBG membership includes "countless" offers to various different things, including discounted hotel rates (not mentioned on their website). The sports teams they offer discounted tickets to see include the New York Yankees. And they have deals with several movie theater chains, including Regal, AMC, and United Artists (UA), to offer discounted movie tickets to their movies in general -- it's not just "special offer: see the new Zappa documentary for $6". (I assume that there are exceptions, e.g., you can't use the discounted tickets to see certain blockbusters on opening weekend; when I've gotten discounted movie passes in the past, that's how it's worked.)

I think my cell phone glitched and ended the call before I could probe further. I am kinda averse to deliberately signing up for a for-profit marketing-centric organization's services in the hopes of ill-defined rewards, so I poked around a bit more.

EBG owns a bunch of sites (why not? "Our Technology Delivers Fun Most Efficiently") so I decided to poke around those on the theory that they're probably giving all the members access to mostly the same experiences, just branded differently and segmented at slightly different price points. Like, their site NewYork.com (available to the public) has Les Misérables tickets for $83 and up, while Working Advantage (companies contract with EBG for member-only discounts) mentions Les Mis orchestra seats for $73 on their front page right now.

Some specific prices and offers: a video urging companies to sign up mentions The Lion King, Walt Disney World, Universal Studios, and Kennedy Space Center as attractions for your employees, and promises prices "up to 50% off what the public is paying". The Tickets At Work blog promises 50% off select Yankees games, or 20% off a luxury suite at a Yankees game. The Broadway shows NewYork.com handles have a lot of overlap with what you'd get at TKTS at (to my eyeballs) vaguely similar prices, so the member-only prices would probably also be fairly good. And the Working Advantage home page mentions several specific attractions, rental car companies, etc. It also enumerates movie chains they cover:

(That's on the front page, under the "Movie Tickets" hover-to-display menu; not super accessible.)

So overall, I think most IDNYC cardholders who have a bit of disposable income, and who enjoy sports/theater/theme parks/etc. but would like to save a bit of money on those things, would find it useful enough to go ahead and register to get the discounts, despite the privacy/spam implications. Hope this helps others make the decision!

Filed under:


(1) : Vis-a-Vis a Visa: As I was applying for the Points Based System Tier 1, General Migrant visa to the United Kingdom, I had a number of questions that the UK Border Agency website and UK visa application site (a.k.a. Visa4UK) did not clearly answer. The UK Consulate in New York City does not allow personal visits from visa applicants and will not take phone calls with questions about visas; they delegate this sort of stuff to the private firm Worldbridge, a division of Computer Sciences Corporation (CSC). Worldbridge charges money to answer questions via phone and does not offer in-person advice.

If the consulate thinks you're missing a document, or have something else wrong with your application, they usually just reject the application rather than phone you to get it cleared up. So I wanted to get everything right the first time. I ponied up to get a person on the phone to answer some of these questions, so I wanted to put the answers up where anyone could read them for free. All this information is courtesy Carolina of Worldbridge.

Worldbridge also takes questions by web form and returns answers via email, but I'm glad I spent the $12 to get half an hour of live chat. Sample dialogue in the phone call included:

"How should I attach the passport photos?"
"All photos should measure 45mm by 35mm; and be in colour; and be taken against a grey or cream background..." [basically reading from the rather frightening photo guidelines PDF]
"I know that. I'm asking how you want me to attach them. Is it okay to put them in an envelope and paperclip it on?"
"You can just paperclip the photos directly to the application."
"But that seems like it'll damage the photos. Is it okay to put them in a little envelope and paperclip the envelope to the application?"
"Yes, that's okay."

I can only imagine how maddening it would be to ask all these questions via email and then battle a response-bot that spouted vaguely-related boilerplate off websites I'd already found inadequate. As it was, Carolina was astonished when I suggested I might highlight or circle relevant bits of my bank statements to make the important numbers easier to find. Yeah, Worldbridge, I can see how thinking ahead to make the user's experience easier wouldn't come naturally to you.

Obviously I'm not a lawyer or expert, all the tips above are my paraphrasing of advice I got on one occasion from one Worldbridge employee in late August 2009, and I can only wish future applicants luck. Hope this helps.

Filed under:



[Main]

Creative Commons License
This work by Sumana Harihareswara is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.
Permissions beyond the scope of this license may be available by emailing the author at sh@changeset.nyc.