(2) Fri Jan 22 2016 23:07 The Minecraft (And Other Games) Archive Project:
As suggested in the previous Minecraft Archive Project post, I have now completed a capture of the CurseForge family of sites. They host a lot of Minecraft stuff I hadn't downloaded before, including the popular Feed the Beast series of modpacks, lots of other modpacks, mods, and a ton of Bukkit plugins (not really sure what those are or how they differ from mods TBH).
CurseForge also has sites for Terraria and Kerbal Space Program, as well as many other games I haven't heard of or don't care about. I paid $30 for a premium membership and grabbed it all, downloading about 500 gigabytes of images and binaries. This doubles the size of the 201512 capture (though it probably introduces a lot of duplicates).
Here are the spoils, ordered by game:
|Game ||What ||Capture Size (GB)|
|Firefall ||Add-ons ||<1 |
|Kerbal Space Program ||Mods ||23 |
|Kerbal Space Program ||Shareables ||1.8 |
|Minecraft ||Bukkit plugins ||19 |
|Minecraft ||Customization ||<1 |
|Minecraft ||Modpacks (Feed the Beast) || 15 |
|Minecraft ||Modpacks (Other) ||87 |
|Minecraft ||Mods ||33 |
|Minecraft ||Resource Packs ||80 |
|Minecraft ||Worlds ||45 |
|Rift ||Add-ons ||7.5 |
|Runes of Magic ||Add-ons ||1.8|
|Skyrim ||Mods ||6.4 |
|Starcraft 2 ||Assets ||4.7 |
|Starcraft 2 ||Maps ||46 |
|Terraria ||Maps ||4.8 |
|The Elder Scrolls Online ||Add-ons ||<1|
|The Secret World ||Mods ||<1 |
|Wildstar ||Add-ons ||1.7|
|World of Tanks ||Mods ||40 |
|World of Tanks ||Skins ||12 |
|World of Warcraft ||Addons ||48 |
Here's the really cool part: CurseForge projects frequently link to Git repositories. I cloned every one I could find. I ended up with 5000 Minecraft/Bukkit repositories totalling 47 gigs, 103 Kerbal Space Program repositories totalling 6 gigs, and a couple hundred megabytes here and there for the other games. That's over 50 gigs of game-mod source code, which I predict will be a lot more useful to the future than a bunch of JAR files.
These numbers are gloriously huge and there are two reasons. 1. this is the first capture I've done of CurseForge, and possibly the only full capture I will ever do. So I got stuff dating back several years. 2. CurseForge keeps a full history of your uploaded files, not just the most recent version (which is typically what you'd find on Planet Minecraft or the Minecraft forum). Some of the World of Warcraft add-ons have hundreds of releases! I guess because they have to be re-released for every client update. And it doesn't take many releases for a 100MB Minecraft mod pack to start becoming huge.
Anyway, as always it's good to be done with a project like this, so I can work on other stuff, like all the short stories I owe people.
Sun Jan 10 2016 08:36 Minecraft Archive Project: The 201512 Capture:
On December 27th I started the third capture for the Minecraft Archive Project. Previous captures ran in February 2015 and March 2014. This time I collected about 420 gigabytes of material.
Here's the breakdown by what I believe the new files to be:
|Type||Number of files||Collective size|
|Maps (MCPE)||1552||2 GB|
|Resource packs||2137||30 GB|
|Resource packs (MCPE) ||176||172 MB|
|Mods||6082 ||10 GB|
|Mods (MCPE)||1839||1 GB|
|Server records||25923||361 MB|
|Blog posts||6562||129 MB|
This time I think I was able to archive about 60-65% of the maps I saw, compared to 73% in the last capture. Even so, we ended up with 33k new maps in this capture versus 22k in the last one--and I didn't even get the adf.ly maps this time! (Nor will I--it's a huge pain and I'm sick of it.) 2012 was the single biggest year for custom Minecraft maps, and there was a downward trend visible in 2013 and 2014, but it looks like 2015 was really huge.
Couple new features in this capture: I started keeping track of blog posts and server records from Planet Minecraft. Server records are especially important because they usually feature screenshots, and in twenty years those screenshots will be the only record of what those servers looked like.
I've completely given up on the idea of archiving public servers--it's still theoretically possible but it's a full-time job for two developers, so I'd need to get a grant or some volunteer interest from the modding comunity. In fact, a few months ago the multiuser server I played Minecraft on went down, and I don't know whether my stuff is still around. That's life! Gonna archive the screenshots.
The full dataset is now about 2.4 terabytes. I bought a new drive to store the archive and set it up with XFS, and it does seem to improve the performance when iterating over the file set.
As always I'm putting a copy of the data on a server at NYPL Labs, and I recently gave Jason Scott a drive that contained the first two captures, so he can do whatever Jason thing he wants with the data. I don't have any plans to make this archive public, or even to re-run the Minecraft Geologic Survey on the new data. My maximum supportable commitment is spending some time once a year to shepherd these scripts through saving a representative sample of this artform.
I'm going to leave everything else to the future when the archive becomes valuable to other people. I am doing exploratory work for adding a third site to the archive, but that's all I'll say about that for now.
Thu Jan 07 2016 08:07 The Crummy.com Review of Things 2015:
Another year has gone, but what's the big deal? Let's remember the magical moments, like 12:12:12 on 12/12, or June 30th's leap second. Good timestamps, good timestamps. Here are the most worthwhile investments of my hard-earned 2015:
I've been giving books short shrift by only mentioning a single Crummy.com Book of the Year, and in 2015 I started reading books on my commute (partly because I'm developing a tool that helps people read books on their commute), so I can afford to mention more than one. I have records of reading 25 books this year, and probably a couple more slipped through the cracks, but I've got a solid best-of slate.
The 2015 Crummy.com Book of the Year is Dragonfly: NASA And The Crisis Aboard Mir by Bryan Burrough. So much good stuff in that book. If you want to write fictional dingy spacecraft, you can't do better than looking at the dingy spacecraft we've actually built.
- Nightwood by Djuna Barnes (who needs her own NYCB post)
- Jim Henson: The Biography by Brian Jay Jones
- You Can't Win by Jack Black (not that Jack Black)
- The Space Opera Renaissance, ed. David G. Hartwell and Kathryn Cramer (book needs its own NYCB post)
- Mastering the Art of Soviet Cooking, by Anya von Bremzen
- The Long Way to a Small, Angry Planet, by Becky Chambers
Honorable mention to Mallworld by Somtow Sucharitkul, a book that I didn't love, but I was blown away by its inventiveness. In 1982, Sucharitkul crammed Mallworld with all the jokes that would later be used in Futurama.
Saw ninety-one features this year. As always, only films I saw for the first time are eligible for consideration, though that only eliminates three. Here are my must-see movies:
- The Americanization Of Emily (1964)
- Mad Max: Fury Road (2015)
- The Brink's Job (1978)
- Inside Out (2015)
- Sullivan's Travels (1941)
- Sunset Boulevard (1950)
- The Breaking Point (1950)
- The Man Who Shot Liberty Valance (1962)
- Sweet Smell of Success (1957)
- Fantastic Mr. Fox (2009)
- The Parallax View (1974)
- Nightmare Alley (1947)
And this year's bumper crop of "recommended" films:
- The Best of Everything (1959)
- Clueless (1995)
- Wagon Master (1950)
- The Crimson Kimono (1959)
- The Godfather, Part II (1974)
- Desperately Seeking Susan (1985)
- Star Wars: The Force Awakens (2015)
- Inside Man (2006)
- The Grapes of Wrath (1939)
- Kundo: Age of the Rampant (2014)
- Ed Wood (1994)
- How To Marry A Millionaire (1953)
- Brainstorm (1983)
- Invention For Destruction (1958)
Honorable mentions to the burglary in Rififi (1955) and the hotel tour in The Shining (1980). I don't want to sit through the whole movie again but those scenes were awesome.
Looking at the list of my follows I feel like I need to broaden my bot horizons because I love all of Allison's bots (except that damn Unicode Ebooks, which still has three more followers than Smooth Unicode) and I love bots that post images from image collections, and that doesn't seem like a very diverse set. Anyway, here are my faves of 2015:
Didn't play a lot of new video games this year because of the persistent problem with my computer shutting off if I dare to start up a game. I did replace the computer near the end of the year, so there will probably be more games in 2016. In the meantime, the Crummy.com Game of the Year is the super-atmospheric This War of Mine; its only flaw, which it shares with nearly all games, is that it's not roguelike enough.
A couple runners-up and honorable mentions:
- 80 Days
- Mini Metro
I played board games pretty regularly but the only new game I remember is the much-loved "Code Names", which I also think is great.
I'd wanted to do an escape room this year, but put the idea on hold when Sumana wasn't interested. Near the end of the year, though, Pat Rafferty (who now works at an escape room in Portland) invited me to join his room-escaping team, and I
leapt stood up at the opportunity. As part of a crew of six, I helped to repair a drifting spacecraft. It was really immersive, finally allowing me to live the experience of crawling through a Jeffries tube.
My only complaint is the puzzles were free-to-play iOS game-level stuff. I understand why you have to do it that way, since none of us would be able to repair a spacecraft in real life, but it meant that a very immersive exploration experience was constantly interrupted by having to decode some Morse Code or solve cheesy riddles. Same reason I didn't like Myst. I did like the puzzles that made you combine objects.
Stereotypically this section would be called "Going Outside", but all the things I want to talk about happened indoors. In fact, two of them happened in the same building: the Town Hall Theater near Times Square. In fact, all of them, since I moved the escape room to the previous section,
Sumana and I both grew up listening to NPR, and we're both fans of the schticky comedy and down-home existentialism of A Prairie Home Companion (though less ardent fans than we were as teenagers). 2015 was the year I told Sumana (paraphrase) "You know, PHC does shows in New York, and as a project focused around a single individual who has been doing it since before we were born, it might not be around for much longer. We should see it live while we have the opportunity." Sumana was convinced by my airtight logic, and we caught the April 25th show. We had lousy seats but it was fun!
Then, near the end of the year, the PDQ Bach Golden Anniversary Concert Kickstarter was announced. As per previous paragraph, Sumana and I are also fans of Peter Schickele's ur-podcast Schickele Mix, so we went through a similar process, although I ended up going to the concert alone. This time I had a great seat! Beautiful music, lots of laughs, I'm really glad I went.
As you can see from the associated pictures, I lost a lot of weight in 2015. I still have a little more planned, but I'm very close to the impossible-seeming target weight I set in July. I found the Atkins diet to be very effective. I don't think I have a lot of self-control, but I am very, very stubborn, and Atkins lets you substitute stubbornness for self-control.
Because of this I didn't exactly spend a lot of time in 2015 exploring New York's burgeoned restaurant scene, and the Food section will be correspondingly short. However, I want to give a special shout-out to the King of Falafel halal food truck in Astoria. See, most places, if you order a meal without the carby thing, they'll simply omit the carby thing, yielding about 60% of a meal. However, if you order a plate at King of Falafel and ask for no rice, they will fill up the empty space with more meat and salad, and you still get a full meal. Thanks, King of Falafel. Saved my sanity.
Also this sugar-free flourless chocolate cake recipe is good for managing your chocolate cravings. Honorable mention: xylitol.
People say that being on Atkins normalizes your energy level, getting rid of the highs and crashes, and I've found this to be true but very inconvenient, since the highs are where I do all my creative work, and the crashes happen at night, a.k.a. "getting sleepy", or they happen at 2 PM, when I drink some tea, problem solved. Right now I feel like it's 1:30 PM all day. Anyway, if you don't count the amazing work I did going from Before to After, 2015 wasn't my most productive year, since I spent half the year in power-saving mode.
But I did finish Situation Normal, and handed it off to an agent, so the book is officially Not My Problem. I've started work on a new novel, Mine, my take on the classic Big Dumb Object In Space story.
I wrote four short stories: "We, the Unwilling" (a bonus story for Situation Normal); "The Katie Event" (the third in the Awesome Dinosaurs trilogy, which you haven't seen because the second in the trilogy needs a revision); "Worm Hunt" (exploratory work for a novel I probably won't write); and "Only G51 Kids Will Remember These Five Moments", which I think I can sell if I ever get around to sending it out.
I gave three talks of note:
I crafted a fabulous NaNoGenMo entry with a one-line shell script: Alphabetical Order.
Four bots came from my fingers in 2015:
I also breathed new life into Smooth Unicode by implementing beautiful emoji mosaics.
Finally I want to wish all of you readers the best in 2016, and to ask you to tell me what you liked in 2015. or what you're proud of accomplishing. I like other peoples' posts like this (Here's Allison's, here's Darius's), and I think taking a moment at the beginning of the new year to look back is satisfying in a way that can't be matched by the corporate "best of the year" lists that dominate the end of the old year.
(1) Tue Dec 29 2015 17:13 December Film Roundup:
The final Film Roundup of the year! Step onto the red carpet, and... no, wipe your feet first! Geez.
- The Last Blitzkrieg (1959): A weird little war movie that I watched for only one reason: it's the only movie I've ever heard of that features a character named Leonard Richardson. Except that's not really his name! "Leonard Richardson" is an alias the main character steals from a red-blooded American POW to carry out a nefarious scheme.
This movie was nearly interesting--there were some moments when it could have taken a really cool turn, got some dramatic irony or moral ambiguity going, a la The Americanization of Emily. But nope! It's a normal WWII movie that was made fourteen years after the war ended. Bizarre.
- Sunset Boulevard (1950): This is a great movie. Exactly as you'd expect me to say. Classic dark-roast Wilder. 'Nuff said.
- The Breaking Point (1950): The sort of surprise that keeps me coming back to the museum. Sometimes I know a film will be great ahead of time, sometimes for educational purposes I watch a "classic" I don't think I'll like, and once in a while I'm blown away by a film I had no particular expectations for. Such was The Breaking Point. This film rises above popcorn noir by focusing not on the gritty glamor of the underworld but on the corruption of a decent family man. Great, great stuff.
- Bring Me the Head of Tim Horton (2015): Guy Maddin somehow gets a job doing the behind-the-scenes documentary on a Canadian war movie. He does his best to bite the hand that feeds. Some good lines ("A war movie is a funeral with no body.") and great gags. The weird video effects are inspired by 80s VHS movies and video games, and thus I find them less annoying than the usual silent-movie schtick. I think Maddin should stick this film on Vimeo.
- Hitchcock/Truffaut (2015): Pretty educational. It was fun to hear Truffaut be a huge fanboy. If I'd known about the book this film is about, my "French New Wave films are secretly genre films" theory would have gotten off the ground a lot sooner. The things you miss out on by not going to film school.
- The Golden Cane Warrior (2014): Starts out really cool, but the best character (the martial-arts mom) dies in the first act, her kids take over, and the middle of the film is kind of a slog. It comes back together for the big fight at the end. Sumana liked it more than I did.
A character in this movie does the most heroic thing you can do in this sort of movie: she stops a village from being burned down, preventing the traditional "burnt village" scene. The villagers get slaughtered anyway, but a valiant effort.
- Star Wars: The Force Awakens (2015): This was going to be my Christmas movie to see with Susanna, but she had to take care of her new baby, so I saw it with John and the niblings who were old enough for it. This movie provides a good illustration of the difference between Star Trek and Star Wars: J. J. Abrams ruined Star Trek, but he did an excellent job with Star Wars.
It's totally Abrams-friendly! Star Wars is based on action set-pieces and eyeball kicks, not thought experiments. The Star Trek characters are all military officers who serve together, but the Star Wars characters are distinctive archetypes, so it doesn't bog the film down to give everyone their scene. We expect a Star Wars movie to have a megalomanical villain, so it's not a disappointment when it happens every single time. The morality is cut and dried: light side, dark side. You can make the hero fight a giant spider in the second act and it makes perfect sense.
I think Star Trek is an important contribution to human culture, whereas I think Star Wars is a fun couple of movies that got out of hand, but I gotta face facts: the Star Wars movies that actually get made are now better than the Star Trek movies that actually get made. I don't like it, but that's what how Hollywood works.
Anyway, a really fun movie. I'm especially tickled that they made the hypothetical "plumbing contractor who works on the Death Star" from Clerks into a compelling, canonical Star Wars character.
- The Last Picture Show (1971): I guess this is the month where I watched movies that claimed to feature "The Last" of something. Appropriate, as this is The Last time I will watch this film. I'm not going to say this is a 'bad' film, there's a lot of good in it, but it hits too close for comfort (I basically grew up in that town) and I also encountered two of my common bugbears:
One, as I've mentioned before, ninety minutes is kind of my cutoff point. I'll watch almost any kind of film if you can keep it to ninety minutes. If you go beyond that point, I need something compelling, like a plot, or fight sequences, or I get antsy. This film is over two hours long, and...
I'm averse to films that could end at any time. The Last Picture Show is such a film. It has a through-line, sure, but since the point of the film is that life is a stochastic process that just creeps at its petty pace to the last syllable of recorded time, I'm sitting here at minute 90+k unsure if this movie's ever going to be over or what. Whereas Celine and Julie Go Boating has a rough first hour, but by ninety minutes a plot is apparent, and by the two-hour mark you can see what has to happen for the film to come to a conclusion.
Just to end on a positive note, it was nice to see young Jeff Bridges. And if you want a cynical 1970s black-and-white Bogdanavich film about the horrible past that's funny and full of life, check out Paper Moon (1973). That movie's more my speed.
- The Cheap Detective (1978): Rewatch with Beth over New Year's Eve. A classic pre-Airplane! spoof with incredible casting (Peter Falk! Louise Fletcher! Stockard Channing!) that gets a lot of laughs out of its absurd dialogue but isn't the perfect classic I remember, because I mentally edited out the bad/boring/offensive parts.
Hilarious and worth a watch, but not tight enough to be a work of genius. Trying to do The Maltese Falcon and The Big Sleep and Casablanca all at once makes it less a tightly focused experience like Airplane! and more like the omnibus spoof movies that dominated stupid comedy in the 2000s. I do think this movie is funnier overall than Airplane!, but I prefer verbal comedy to sight gags, and there's no wasted space in Airplane!. Unless I mentally edited that movie as well.
And now, the Television Spotlight focuses on a show that we watched in its entirety
- John Adams (2008): Hamilton-mania continues to run rampant in our household, and I had the idea to apply HBO's recent miniseries to Sumana's forehead as a sort of poultice. We had a good time and enjoyed the subtle shout-outs to last month's poultice, 1776. The John/Abigail relationship is always a winner.
If you look at the reviews for this movie, you'll see that a lot of the low-rated reviews are based on complaints about historical inaccuracies, but they're generally pretty minor inaccuracies, well within the range of... Creative License. In fact, in the final episode, Adams, talking to John Trumbull, makes the 'historical inaccuracy' critique more effectively than most John Adams reviewers, who admittedly may not have made it to the final episode. Just a little bit of fourth-wall breaking to send you on your way.
I haven't read the book but I think this series does a good job of portraying Adams the way he might have seen himself: as an unappreciated figure, always working away in someone else's shadow, a man whose greatest accomplishment as president was having the guts to do nothing when the public was demanding he make a horrible mistake.
Tue Dec 01 2015 22:17 November Film Roundup:
I remember this month's movies being meh-ful, but when I went back to the list there were three really good movies, and I'd just allowed my memories to be overwhelmed by the underwhelming movies, because I saw the three really good movies all in a row. No more! Let joy be unconfined!
- Aparajito (1956): I believe this movie was bankrolled by Indian moms looking for effective ways to guilt-trip their children. I saw this while Sumana was out of town. Sumana really wants to watch the Apu trilogy, and I'm happy to watch these movies with her, but it's the kind of episodic character study stereotypically associated with foreign film and it's not a great way for me to spend my alone time. PS: Call your mother!
- My Name Is Nobody (1973): An attempt to deconstruct the spaghetti western a la Sergio Leone, the way John Ford deconstructed his own work The Man Who Shot Liberty Valance. I don't think it works very well. Ford's films are full of humor and in Liberty Valance he uses that humor to fuel the dramatic irony. I find spaghetti westerns effective insofar as they're bleak and kinda humorless, and this film pours on the humor to create a satire of the genre. Admittedly this was (barely) pre-Blazing Saddles, so I understand why this movie was made, but between Blazing Saddles on the lowbrow end and Liberty Valance on the highbrow, the western is pretty well deconstructed by 1974.
This movie contains an awesome sight gag involving more pool balls on a pool table than I've ever seen before. I'll always remember that sight gag and I've already forgotten most of the rest of this movie. On IMDB for this film Sergio Leone is credited with "idea", and I hope his idea was "you should do a gag with a bunch of pool balls on a pool table" and not "what if you made a film that exposed the shallow conception of heroism in the western?" Because John Ford already had that idea.
- 1776 (1972): Among movies whose titles are years, the one with the largest delta from the year the movie was made is probably One Million Years B.C. (1966), and the smallest is Nineteen Eighty-Four (1984). In between we have... this fine movie. You may know that Sumana is obsessed with Hamilton, but I don't want to listen to the soundtrack until I've seen the play, so we saw 1776 together as a compromise move because our Hamilton tickets aren't until next year.
Sumana found it a learning experience since 1776 was a big influence on Hamilton. We agree that it's incredibly ahistorical and that the songs are overall not great (Sumana: "Do we really need a song about how Jefferson plays the violin?"). The villains (i.e. the Southern reactionaries) have the best songs, like the one that exposes New England's complicity in the slave trade. Howard da Silva does a great job playing Benjamin Franklin as I've always pictured him: as America's wacky Falstaffian uncle. According to IMDB da Silva also portrayed Franklin "in a National Park Service film presented in the 70s and 80s at Ben Franklin's home at Franklin Court, Philadelphia, Pennsylvania," making this also the federal government's official portrayal of Franklin.
- Johnny Guitar (1954): Didn't find it that enjoyable, and in retrospect I mainly wanted to see this because everyone in the screenshots looks like Jeff Goldblum in Buckaroo Banzai. Not a good reason to see a movie. I did like Joan Crawford being real brassy. Just a hunch, but I think this movie is a lot better if you're a forty-year-old gay man. I feel that's the approximate shape of the thing I don't understand here.
- Out of the Past (1947): Stereotypical film noir with a character named Leonard! Boosts my hypothesis that Leonard is a perfect film noir name. For heroes, villains, thugs, cops, society gents or skid row bums... "Leonard" always works. Consider naming your next noir character Leonard!
Oh yeah, everything else about the movie. The first few scenes defied convention with their setting and mood, but it settled in to the familiar pathways pretty quickly. Overall... popcorn noir, recommended, but not highly.
- Nightmare Alley (1947): Now this is some noir. It starts at a carnival, the place where all happiness is false and misery is paraded as entertainment. And it all goes downhill from there, and you're along for the ride. Great stuff. In particular the portrayal of a ruthless woman psychiatrist who sleeps in a Ruth Bader Ginsberg outfit seems unusually progressive for 1947.
- Clueless (1995): One of Sumana's all-time favorites, and an entry on our "women directors" watchlist. I saw it for the first time this month and I gotta say this is a great movie. The characters change over the course of the film, they avoid being John Hughes teen stereotypes, and the only real villain is dispatched pretty early on, allowing for plenty of conflict that's not predicated on someone being the antagonist. Lots of laugh-out-loud moments for me.
One weird thing: the cell phone jokes don't land anymore. You can see them happening, you know they are jokes, but everyone has cell phones now so the jokes don't do anything. They're like the ghosts of jokes.
- The Crimson Kimono (1959): Wow, what an unusual movie. It's resembles noir, but it's too procedural, too earnest, and it has a happy ending. None of the cops are crooked; they just have personal problems that get in the way of their work. It goes overboard showing that Japanese-Americans are good, patriotic Americans. In general, it has too much faith in humanity to be film noir. But it was more daring in its time than more cynical movies, and it's the rare movie that makes me want to seek out more of this director's vision. Really glad I saw this one.
- It's the Old Army Game (1926): At this point I gotta say that W.C. Fields, like Jerry Lewis, is one of those comic legends I just don't find funny. A misanthropic loser can be a hilarious character, but I only laughed at some of the physical comedy (like the Stooges but more highbrow). The best thing about this movie was that the Zeppoish love interest resembles Derek Waters from Drunk History, allowing me to pretend that the whole thing was a Drunk History vignette gone wrong.
This silent film includes a title card containing must be the ultimate W.C. Fields line: "I'll hit him in the face with this kid!"
- In honor of Clueless, this month the Television Spotlight focuses on Square Pegs (1982), a really smart television show about high school girls, created by SNL writer Anne Beatts. It's clever and funny in the same way as Clueless, but it's even better because it focuses on the misfits rather than the popular kids. Watch it today! Includes Devo.