Categories: sumana | Transparency in Government Code
Algorithms and software governments make and use should be available for public inspection and reuse
# 01 Jul 2021, 12:58PM: Software Bill of Materials & the US Federal Government:
In February, the United States's President Biden signed an executive order on the US's supply chains; he followed this up with an EO in May specifically concentrating on improving cybersecurity. To quote Tidelift's summary, "in essence, this order is a striking attempt to create a new global standard for cybersecurity that all organizations around the world will need to ensure their software supply chain meets or exceeds in the near future."
Because of the EO, the US's National Telecommunications and Information Administration requested comment "on the minimum elements for a Software Bill of Materials (SBOM), and what other factors should be considered in the request, production, distribution, and consumption of SBOMs". I heard about this thanks to Jacob Kaplan-Moss, who suggested the Python Software Foundation could share its open source perspective. So I led an effort to submit a comment on behalf of the Packaging Working Group of the PSF - thanks to Morgan Mayo (PSF Director of Resource Development) for some of the prose!
The Packaging WG comment, along with comments from 80+ other individuals and groups, are now up at the NTIA website. Check out Section II ("Background context about the Python packaging toolchain and ecosystem") for a simplified yet still confusing diagram. Section IV of our comment ("Infrastructure funding") is pretty short; for a longer treatment on a related topic, see Tidelift's comment "Software bills of materials are important—but they won't work at scale if we don't pay the maintainers".
# 06 Feb 2020, 10:27AM: My First Exascale Computing Project Annual Meeting:
Some interesting things about attending the Exascale Computing Project Annual Meeting for the first time, and stuff I have learned here so far!
[Edited 1:10pm CT to add: By the way, here is a contextual note for people who don't usually read my blog. I'm Sumana Harihareswara, a project manager and open source consultant who hadn't heard of ECP before November, and who primarily works in Python and outside of government stuff. I haven't done any kind of systematic survey of all ECP participants/attendees so these are my impressions based on people I've talked with and talks I've attended.]
- Here is the overview of the Exascale Computing Project, which started a few years ago. Giant high-performance computing hardware, software, applications, training, and so on, working a lot at the United States's National Laboratories (like Lawrence Berkeley, Oak Ridge, Argonne, Los Alamos, and so on). Thus there is a lot that is public (for instance, see this report on improving scientific productivity, or this capability assessment), but then there are talks I'm not allowed to attend because I haven't signed the relevant nondisclosure agreement.
- They contribute a bunch to LLVM and to Spack, a package manager. There are like 6-7 full-time funded people working on Spack [Edited 1:11 CT for correction: no, this is more like 6-7 people who work full-time and who spend at least a chunk of their time on Spack], and dozens of people attended the Spack state of the project/feedback roundtable session. Researchers and developers within ECP are working on a bunch of open source projects (example), some extremely specific to high-performance computing math things, but some more generally useful tools, and many folks in the project would like to get broader publicity and adoption for the latter. There are some opportunities here for cross-pollination, funding, user testing, and de-duplication between work being done by DoE and work being done in the larger open source industry.
- The ECP is sponsored by the US Department of Energy. And, you know, that means fossil fuels too. There's an Industry Council and ExxonMobil is on it. The National Labs do a bunch of work for DoE and other US government departments -- and
for the private clients who can afford it [Edited 1:23pm to correct this; those orgs aren't paying the labs to do work, they're getting to use the facilities just like anyone else could (example)], which is often the fossil fuels companies who want to run simulations having to do with oil and gas. When I've talked to folks here about how that feels weird, I get a variety of responses. Some people point out that there is a National Renewable Energy Laboratory among the ECP Participating Labs, or that the combustion work in the labs helps energy companies figure out how to use gas more efficiently so we burn less fuel, and so on. One person basically said: They're an important industry and it's part of our job to help them; it's the Department of Energy and that means all energy. Another person basically said: As soon as feasible, I want us to not do that work anymore.
[Edited 1:12pm CT to note: of course these are my personal observations and not a "here is an official position" thing.] I don't think anyone here denies that climate change is happening. I think they're supposed to make an attempt to not use that phrase in official published materials and they're not supposed to talk about it when they go to DC, though. In one talk a speaker mentioned that one of the categories he was listing was "Earth and Space Science -- what we used to call climate." I said, "Sorry, I'm new. What do we call it now?" and got the answer: "Earth Systems."
- Weapons! Yeah the DoE includes the Office of Science (SC) and the National Nuclear Security Administration (NNSA). And the National Labs do some work for the military, the Department of Homeland Security, National Security Agency, and so on. Also there's some back-and-forth where sometimes people, for instance, start at the Department of Defense and then start working within DoE.
Approximately everyone at this meeting is fine with the fact that some of their work (or maybe a lot of their work) has to do with weapons. [Edited 1:16pm and 4:50pm CT to say: so I've been told that this is mega inaccurate and that a buuuuuuunch of people's work here has NOTHING to do with weapons, is just pure open science, that there are several labs where nearly no one directly works on weapons stuff, or that there are several labs where no one does. Also I've been pointed to the DoE budget where only a fraction of the yearly spend goes to NNSA labs, and those labs also do a bunch of open science research. I need to look into this more to understand the nuances. Also, it was pointed out to me that, if I'm saying "this work is not directly weapons work but it is foundational to weapons work," then, one could also justly say that my work in Python also supports weapons research. Yup, it sure does! I am definitely complicit in things I am uncomfortable with! It's complicated.] Again, some people, when I bring this up, point out how much of the work has nothing to do with weapons, or talk about the work of stockpile stewardship as being primarily about safekeeping of and knowledge transfer about nuclear warheads where there is no likely near-term path to the US completely getting rid of them, or talk about defense in a world where nukes are out there and not about to go away. And at least one person said, basically, I have no problem with the weapons stuff and it's cool.
- The vast majority of people here have doctorates, usually in one of the mathematical, computational, or physical sciences. I haven't seen a single name badge that has "Dr." on it; I think it would take up room and seem egotistical. Also, I am very rarely the only woman in the room, and some of the leadership are women, but I'm often the only person in the room who doesn't know C (or Fortran; the software ECP is writing for or adapting to the new machines is basically 2/3 C and C++, 1/3 Fortran). So my particular configuration of insecurities this week is different than it often is at tech conferences.
- I am, here, extremely unusual in that I do not work for Department of Energy, one of the National Labs, a university, or a big company that is in the Industry Council. People squint at my badge, which says "Sumana Harihareswara, Changeset Consulting," and ask "where are you from?" And I say "New York City" and they say, "Oh, Brookhaven?" and then I explain that I'm a Better Scientific Software Fellowship Honorable Mention and that I'm working on materials to help people maintain open source software better. On the second day of the conference, I took a pen and added "BSSw" to the badge to help jump-start this process.
- People here will refer to "a code" to mean an application or a particular simulation, where I might say "a tool". A person might refer to "running industrial codes" or "legacy codes that have been used for decades".
- One of the kinds of sessions I'm not allowed in is the detailed PathForward stuff; DoE is contracting with chipmakers to do research and development and get big cutting-edge supercomputers for the ECP.
Following a rigorous review process, six responses were selected for award and contract negotiations began. All six selected responses successfully led to contracts that were awarded and announced in June 2017. The six awardees were Advanced Micro Devices (AMD), Cray Inc. (Cray), Hewlett Packard Enterprise (HPE), International Business Machines (IBM), Intel Corp. (Intel), and NVIDIA Corp. (NVIDIA).
HPE has bought Cray so that reduces the competition among these vendors -- and the redundancy in case one of them delivers late, goes bankrupt, or what have you.
- Some people who are not US citizens work at the National Labs, including the more weapons-centric ones. [Edited 1:13 CT to note: I said "many" originally, but this is not to say that non-US-citizens are a majority! There are thousands of people working at the National Labs; "many" does not mean "most," just, like, there are some. I don't have exact numbers here and am changing "many" to "some".] They are open to hiring people from other countries. Also, National Labs employees are kiiiiiiiinda US government employees and kinda not in a way that I don't understand well enough to explain. But there are national security projects within the US government that would appreciate if more US citizens got into science and engineering research -- hence, for example, the National Science Foundation Graduate Research Fellowship Program (GRFP) which
helps ensure the vitality of the human resource base of science and engineering in the United States and reinforces its diversity. The program recognizes and supports outstanding graduate students in NSF-supported science, technology, engineering, and mathematics disciplines who are pursuing research-based master's and doctoral degrees at accredited United States institutions.
Fellows share in the prestige and opportunities that become available when they are selected. Fellows benefit from a three-year annual stipend of $34,000 along with a $12,000 cost of education allowance for tuition and fees (paid to the institution), opportunities for international research and professional development, and the freedom to conduct their own research at any accredited U.S. institution of graduate education they choose.
And they don't require a GRE score, by the way. Also you can sign up to help review applications!
- The researchers at the National Labs, like a lot of scholars within academia, care about getting papers published, and sometimes that gets in the way of good maintainership for their open source projects. For instance, if you are worried that sharing your feature roadmap for your open source tool will let someone else get the jump on you and get a paper submitted sooner, you might hold that information kinda secret, which makes it more likely users will duplicate that work in their own forks.
- The different National Labs have different cultures and "the further they are from a city, the weirder they get".
Thanks to BSSw for bringing me here! [Edited 4:57pm CT to add: I went on so long about these pseudo-anthopological observations that I need to start a new entry about cool tools I found out about here! Hope that will be next.]
# 19 Nov 2018, 08:23PM: The Fascination Of Municipal Taxation Software:
On January 30, 2018, I attended my local city councilmember's "State of the District" speech. If one of your local officials offers such a speech, I recommend going; it's a structured way to find out what issues they think are important. And Councilmember Constantinides scheduled this one the same night as the US President's State of the Union address, which felt like a welcome alternative.
Among the plans and promises that got public notice, he mentioned a government IT project he wanted New York City to implement. In his address, Constantinides said he was introducing a:
bill that will direct the Department of Finance to create a website where anyone can view their property tax exemption status. Under this new website, property owners would be able to pay their taxes, directly submit questions to the DOF, and view their records. They'd be able to access specific information regarding their properties including applications for exemptions like the Senior Citizen Homeowners' Exemption, status of exemptions, date by which they'd need to apply to renew an exemption, or whether anything has expired in their record. If a property owner's application is rejected, they must tell you why. Property owners will also be able to set up alerts for any changes.
These are simple, common sense things that already exist on other government platforms, and the fact that the Council may have to pass legislation to create this system is very disappointing. But if we're going to ask you to pay substantial sums of your hard earned money to fund the government, the government needs to uphold its end of the bargain and give you all the tools it can to manage your payments.
The proposal caught my attention because I find it inherently interesting (and kind of amusing) when politicians give speeches about web apps. I took the photo because I couldn't remember the last time that a politician, literally giving a speech from a podium near a US flag, presented a functional spec for software he wanted, in bullet points. I'm a project manager and a programmer who has worked on multiple software projects for local governments. Some part of me, for a fraction of a second, saw that bullet list and thought, "OK, that's the scope. How many programmers do I have and what technologies will we be using?" before remembering that this was not my job.
Constantinides mentioned the bill again in a spring newsletter and I dug around a bit. Introduction 0627-2018: "Establishment of an online system to access property tax information and receive notification of changes to property tax exemptions." The Council referred it to the Committee on Finance, which hasn't held any hearings about it yet.
On the one hand, getting the local government to make a web application for property tax stuff makes obvious sense (and other localities, such as Santa Clara County, already do it). Public servants need to help the public, and so much of public service requires software. On the other hand, government IT projects have such a bad reputation. Ten years ago, Dan Davies wrote: "nearly anything new that the government does is going to require an IT element ... government projects tend to only come in one size, 'big', and to very often come in the variety 'failed'." I inhale sharply when I see someone propose a new government IT project, because I instantly foresee manifold hazards.
But we know a bit more than we did ten years ago about how to address those concerns. There's vendor lock-in, which is a big reason to prefer building or reusing open source applications. There's metadata wrangling and legacy application/infrastructure compatibility, and partnering effectively with agency staff -- 18F and the US Digital Service have grown serious capabilities in those areas. There's the challenge of serving everyone, no, seriously, everyone ("government doesn’t always appear to provide a satisfactory solution is because government has to take on the hardest problems") -- and we can incorporate "no, seriously, everyone" into our design processes....
And that last point -- about how government needs to serve everyone gets at perhaps the deepest reason this proposal caught my attention. I used to be incredibly interested in taxation, to the point where I considered following in Dr. Robin Einhorn's footsteps and going into the academy to seriously research tax history. And a big reason is that taxes affect everybody, often noticeably. A resident might be pretty oblivious to all the other ways government activities touch their life, but a ton of taxes impinge on their perception and cause notice -- income taxes, use fees, sales/value taxes, property taxes, payroll taxes, &c.
Taxes are surface areas, user interfaces, where the least-informed user unavoidably comes into contact with your system, and notices it, and (mostly) inherently resents the cost you're imposing on them, and thus finds any friction along the way particularly maddening. This reminds me of something Leonard wrote in 2003:
You can pay your San Francisco parking tickets online. This makes sense, as the general philosophy of the city of San Francisco is to make it easy for you to deal with the arbitrary aggravations they inflict upon you.
And, just like with software interfaces, tax structures have these nasty path-dependent ways of accidentally creating interest groups. Randall Munroe's xkcd #1172 ("Every change breaks someone's workflow") obscurely reminds me of Conversion and the Poll Tax in Early Islam by Daniel Dennett, Jr. -- if you use reduced taxes as an incentive for some behavior, such as conversion to Islam, and then people do that and your tax receipts go down, and then you try to make up for losses by raising taxes on the folks who now feel entitled to a tax break, the interest group you have just created will grumble or rebel.
And maybe this lens helps explain why I bang on about the governance side of maintainership, and how a bug tracker anyone can report issues to is a sign of hospitality and humility and stewardship, and and so on. Every once in a while a stranger calls me a politician. I'm not seeking elected office and I'm only as accountable to my neighbors as they are to me. But I am attuned enough to socially constructed things that I notice and try to work with them, and I try to notice where the resources come from and where they go and who ends up getting taxed, and how.
# 16 Oct 2018, 11:22AM: NYC Comptroller Town Hall, And Reflections on Constraint:
Last night I suited up and went to a local town hall held by the office of New York City's Comptroller, Scott Stringer. (I am in the fuzzy foreground of the second photo.) After very short introductions from the venue host (CUNY Law School), Stringer and his staff, we went straight to questions!
I appreciated a lot of things about the event. There was an ASL translator on the stage, and when residents wanted to ask questions in Spanish, a staffer translated between Spanish and English for them. Stringer kept the lines moving by answering folks' questions but also limiting them to one question each (or they could head to the back of the line to get another turn), and interrupted rambly rants by asking for a question he could answer. And if people spoke up with complaints, he promised: fill out a constituent intake form and give it to one of my staffers, and we will call you by noon tomorrow. And free bottled water, next to the paper copies of audit reports and outreach flyers, was a nice touch.
I asked the first question: how can we save money in IT procurement? Perhaps by banding together in consortia with other municipalities to have better leverage with vendors, or making or using open source software? I fear I was not very clear and was misunderstood. Stringer replied by talking about the need to modernize the procurement process itself, which is evidently still paper-based and slow, and about how this depends on revising the City Charter. Wendy Garcia (the office's Chief Diversity Officer) followed up by suggesting that I myself might want to come to their office so they could help my business figure out where our services matched up with the city's contracting needs. [I spoke with her after the town hall to clarify: no, I'm not trying to get business for Changeset here, I'm just interested in the issue! (Maybe I misguided them by introducing myself as a consultant and wearing a suit. The suit was just to respect the occasion! Next time maybe I will wear a stylish dress and cardigan, which seems to be what middle-class women activists wear to these things??)]
I filled out a constituent intake form, and, sure enough, just before 10:30am today, I got a call from their office asking me to email a specific staffer with more details! Well done.
Other questions and answers included a wide variety of concerns: older guy who doesn't like streets getting named after politicians, frequent meeting questioner guy whose stuff was taken (and never returned) when he was arrested in 2015, the Major Capital Improvement rule landlords use to get around rent control, Department of Education buildings that perhaps ought to be reused instead of sold, divesting NYC's pension fund of fossil fuel, Stringer's political ambitions, an idea for stop sign speed sensors (like traffic light speed sensors), the closure of the jail on Rikers Island, helping immigrants pay the costs of applying for citizenship, sewer problems, the placements of homeless shelters, and helping residents use their on-time rent payments to count towards credit scores. My neighbors care about a lot of different things. I took a few notes and mostly sketched. There was this one power outlet mechanism embedded in the desk right in front of me and I drew it like five times and never got the angles to look right.
One interesting thing I learned: when the Comptroller's office audits a city department, it usually takes about 18 months, so they only go in and do an audit if they think it's likely they'll find something.
I went home and commented on the proposed National Park Service rule change "Special Regulations, Areas of the National Park System, National Capital Region, Special Events and Demonstrations". I commented on 4 things: making the swimming/wading rules more consistent, removing the "duplicative" criterion, the "atmosphere of contemplation" expansion, and the proposed permit application fees. And then I wrote a thing to prepare for a meeting today, while texting with a friend who's going through a rough time.
I don't know anyone who's not going through some kind of rough time. Or at least I can't think of any. If nothing else we have the awful "well, MY life is great, but the world is horrifying" awareness; it feels like we're betraying our neighbors when we enjoy our personal successes. I never know whether I'm doing enough; I have to define "enough" for myself, which feels audacious. Willow Brugh wrote about how she's implementing a concept I first heard about from Abi Sutherland in December 2016:
While I am pushing to find ways to gain (and deserve) greater influence in the world, those things which fall outside of my influence cannot be that which concerns me most. To do otherwise is a path to madness. I must trust that other capable people exist in the world, and that they are taking up their share just as I am taking up mine. As you are taking up yours.
# 17 Dec 2017, 11:02AM: Bill 1696 and Learning Old Systems:
A very amended version of Councilmember Vacca's algorithmic transparency bill has now passed the City Council and is headed for the Mayor's desk to sign.*
This follows the October 16th hearing (which was moved to a larger hearing room at City Hall due to huge attendance) -- the video recording is available now and is a little over two hours long, as are PDFs for Hearing Testimony (pre-written) and the hearing transcript. One attendee live-tweeted practically the whole hearing (sometimes the threading broke a bit) and another shared rough notes as a GitHub gist. Several people spoke for a few minutes each from, e.g., the New York Civil Liberties Union, The Brennan Center for Justice, Legal Aid Society, BetaNYC, Brooklyn Defender Services, Princeton's Center for Information Technology Policy, and various other institutions, and some spoke just as individuals. I testified for a few minutes, starting at about 1:53 in the video, and got quoted in Civicist.
The amended bill, approved by the Council's Technology Committee and then by the City Council earlier this month, is a compromise. It creates a task force, and they'll have 18 months to write up a report with recommendations, and that report will be made public. The bill specifically says that "Nothing herein shall require compliance with the task force's recommendations". Who appoints the members of the task force? "Such task force and the chair thereof shall be appointed by the mayor or a designee thereof" with no particular mandate that, say, the Council has a voice in who's placed on the task force. The bill says nothing about whether the task force will perform any of its hearings in public.
So those of us who want to keep momentum going on this issue will have to note who's been appointed, submit testimony when the opportunity arises, and find a way to sustainably pay attention to it.
The Mayor could allow the bill to lapse into law without signing it, could sign it into law, or could veto it (and then probably have the Council override his veto with a two-thirds majority). What I hear is that it'll almost certainly be the first or the second of those three. Legistar says there'll be a hearing tomorrow, Monday the 18th, but what I've heard is that this will be kind of a formality in which 20+ bills are being "heard" but no substantive discussion is expected.
So I tried to find out when on Monday this hearing will be. I looked around the Mayor's chunk of nyc.gov and found nothing. My Council contact told me that the daily Sked in the daily First Read in City & State and Gotham Gazette's Week Ahead sections would tell me these kinds of schedule details -- once Monday rolled around.** Ahhh. New York City is a very old system, like sewing or software packaging,*** and way before there existed a municipal website, there was a rich ecosystem that depended on knowing this information, and so niche publications emerged. Right.
And today, while writing this, I found the City Record Online (every day the City Record puts out notices of city hearings, court notices, etc. and you can look at recent daily editions as PDFs or search electronically), and figured out: 4:30 pm, in the Blue Room at City Hall, as announced on December 13th.
So I'll probably be there, even though it probably isn't substantively important, as I learn this system, as I learn how to pay attention. Maybe I'll see you there too.
* Legistar, the application that NYC uses to track bills as they move through the City Council, has email and RSS notification, but the email alerts have not been functioning for me, and the RSS option is pretty uninformative and (I think) slow to update. Councilmatic is an open source alternative that had to use screen scraping to get bill and event data (the comments in the bills scraper elucidate some stuff I'd been unsure about). I'm glad to hear that, thanks to NYC open data advocates, there's now a proper Legistar API available for civic developers like us.
** Indeed, the First Read now includes a sked for Monday that mentions a Mayoral hearing and bill signing -- but doesn't specify or link to the list of bills.
*** I'm improving various skills and learning multiple systems right now. In rough order of how old our systems/skills are, as humans, here are some of them:
- Time management (regular)
- New York City governance
- Sewing (electric)
- Time management (with mass media)
- Bicycling (in car-heavy urban environments)
- Time management (with email)
- Python packaging
- Time management (in attention-casino electronic environment)
You can hire me through Changeset Consulting.
This work by Sumana Harihareswara is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.
Permissions beyond the scope of this license may be available by emailing the author at firstname.lastname@example.org.