Blog by Sumana Harihareswara, Changeset founder

26 Sep 2016, 9:33 a.m.

iCalendar Munging with Python 3, Requests, ics.py, and Beautiful Soup

Hi, reader. I wrote this in 2016 and it's now more than five years old. So it may be very out of date; the world, and I, have changed a lot since I wrote it! I'm keeping this up for historical archive purposes, but the me of today may 100% disagree with what I said then. I rarely edit posts after publishing them, but if I do, I usually leave a note in italics to mark the edit and the reason. If this post is particularly offensive or breaches someone's privacy, please contact me.

Leonard and I love seeing movies at the Museum of the Moving Image. Every few months we look at the calendar of upcoming films and decide what we'd possibly like to see together, and put it on our shared calendar so we remember. And for every showing (example) the MoMI provides an iCalendar (.ics) file, to help you add it to your electronic calendar. But it's a pain to individually download or refer to each event's .ics file and import it into my electronic calendar -- and the museum's .ics files' DTEND times are often misleading and imply that the event has a duration of 0 seconds. (I've asked them to fix it, and some of their calendar files have correct durations, but some still have DTEND at the same time as DTSTART.)

Saturday morning I had started individually messing with 30+ events, because the MoMI is doing a complete retrospective of Krzysztof Kieslowski's films and I am inwardly bouncing up and down with joyous anticipation about seeing Dekalog again. And then I thought: I bet I can automate some of this tedious labor!

bash terminal showing the successful output of a Python script (a list of movie titles and "Calendar ready for importing: MoMI-movies-chosen-2016-09-26.ics") So I did. The create-fixed-ics.py script (Python 3) takes a plain text file of URLs separated by newlines (see movie-urls-sample-file.txt for an example), downloads iCalendar files from the MoMI site, fixes their event end times, and creates a new unified .ics file ready for import into a calendar. Perhaps the messiest bit is how I use a set of regular expressions, and my observations of the customs of MoMI curators, to figure out the probable duration of the event.

Disclaimers:

  • It can be a bit slow as the number of URLs adds up -- it took maybe 5 minutes to process about 31 events. I oughta profile it and speed it up. But I usually only need to do this about six times a year.
  • This script is not careful, and will overwrite a previously created .ics file at the same address (in case you're running it twice in one day). It has no tests and approximately no error-checking. This was a scratch-my-own-itch, few-hours-on-a-Saturday project. No Maintenance Intended. 'no maintenance intended' badge
  • Absolutely not an official project of the Museum of the Moving Image.
Much thanks to the programming ecology that helped me build this, especially the people who made RegExr, Beautiful Soup (hi Leonard), Requests, ics.py, and the bpython interpreter, and the many who have written excellent documentation on Python's standard library. Thanks also to Christine Spang, whose "Email as Distributed Protocol Transport: How Meeting Invites Work and Ideas for the Future" talk at Open Source Bridge 2015 (video) introduced me to hacking with the iCalendar format.