Intended Effect

A GitHub Copilot AI Anecdote
I’ve been watching Three Body—the 2023 Chinese television adaptation of the same Liu Cixin science fiction novel that Netflix also put out a version of. My copy was found on the high seas shortly after the release of the series, but it’s now available on a few streaming services. I was about 20 minutes into watching Episode 13 when the subtitles suddenly lost their sync with the video; text appeared on screen about 15-20 seconds after it was spoken. I tried Plex’s feature for downloading alternate subtitles, but each one exhibited the same problem. I spent $2 to just buy that episode from Amazon, skipped ahead to where I had left off, and even that had the same misaligned subtitles. This post is the story of how, having failed to buy my way out of the problem, I used AI to fix it.

I have a subscription to GitHub Copilot through my work, so after a rough assessment of the problem I opened the offending srt caption file in Visual Studio Code and submitted this prompt in the chat window:

The srt file I have open is incorrect; all captions are set 15 seconds too late beginning with caption number 225. Please correct the srt file, altering the time codes for all caption entries beginning with 225, to have them appear on screen 15 seconds earlier.

What I wanted was for the robot to just echo out a repaired srt file, but instead it offered a Python script that could fix the problem. Apologies for the spoiler, but anyone skimming this post for a solution to their own caption file problems might want to skip a bit farther down:

To adjust the time codes in an SRT file, you can read the file line by line, identify the lines that contain time codes, and adjust the time codes as needed. Here’s a Python script that does that:
```
import re
from datetime import timedelta

# Open the SRT file
with open('Three-Body.S01E13.CHINESE.WEBRip.x264-ION10[eztv.re].srt', 'r') as file:
    lines = file.readlines()

# Regular expression to match time codes
time_code_regex = re.compile(r'(\d{2}:\d{2}:\d{2},\d{3}) --> (\d{2}:\d{2}:\d{2},\d{3})')

# Time delta to subtract
time_delta = timedelta(seconds=15)

# Adjust the time codes
for i in range(len(lines)):
    # Check if the line is a time code line
    match = time_code_regex.match(lines[i])
    if match:
        # Check if the caption number is 225 or higher
        caption_number = int(lines[i-1].strip())
        if caption_number >= 225:
            # Parse the start and end times
            start_time = datetime.strptime(match.group(1), '%H:%M:%S,%f')
            end_time = datetime.strptime(match.group(2), '%H:%M:%S,%f')

            # Subtract the time delta
            start_time -= time_delta
            end_time -= time_delta

            # Format the new time code
            new_time_code = f'{start_time.strftime("%H:%M:%S,%f")[:-3]} --> {end_time.strftime("%H:%M:%S,%f")[:-3]}'

            # Replace the line with the new time code
            lines[i] = new_time_code

# Write the adjusted lines back to the file
with open('Three-Body.S01E13.CHINESE.WEBRip.x264-ION10[eztv.re].srt', 'w') as file:
    file.writelines(lines)
```
This script opens the SRT file, reads all the lines into a list, and then iterates over the list. For each line, it checks if the line is a time code line and if the caption number is 225 or higher. If both conditions are met, it parses the start and end times, subtracts 15 seconds from each, and replaces the line with the new time code. Finally, it writes the adjusted lines back to the file.

Please replace 'Three-Body.S01E13.CHINESE.WEBRip.x264-ION10[eztv.re].srt' with the path to your SRT file if it’s different.

The script looked decent enough at a glance, although it didn’t include any details about how to run the script. I knew to put it in a .py file, and although I have very little experience with Python I could guess that the script might need to be run with python3 rather than python on my Mac laptop (which turned out to be true, but perhaps due to some quirk of my own setup). The script also edits the file in-place, rather than saving the result out to a new file, which I would consider bad practice; arguably it should take the srt filename as an input parameter, but then again we’re just trying to solve a single, weird, one-off problem. So we’ll deduct a couple of “ease of use” and “best practice” points, but more importantly the script didn’t work!

First, it failed to import the “datetime” module—the syntax highlighting in VS Code made that obvious enough, but the IDE’s proposed fix of import datetime wasn’t correct, either. So I added from datetime import datetime to the top of the file and ran the script against my local copy of the srt. I didn’t bother to look very closely at the result—in my defense, it was 10:30pm and I was in “watching tv” mode, not “I am a software engineer” mode—and copied it to the Plex server. I restarted playing the episode where I left off and… now there weren’t any subtitles at all!

Let’s look at a snippet of the edited srt file to see if we can spot the problem:
```
223
00:16:25,660 --> 00:16:30,940
Correcting them is what I should do.

224
00:16:31,300 --> 00:16:45,660
At that time, I thought my life was over,
and I might even die in that room.

225
00:16:51,020 --> 00:16:54,500This is the person you want.
I've handled all the formalities.

226
00:16:54,500 --> 00:16:56,420You know the nature of this, right?
```
Subtitle formats are actually pretty easy to read in plaintext, which I appreciate. And a glance at the above snippet shows that GitHub Copilot’s script resulted in the timestamp of each entry running immediately into the first line of text of that caption. I’m trying to keep this relatively brief, so I’ll just note that a cursory search turned up a well-known quirk of Python’s file.readlines method (which reads a file while splitting it into individual lines of text), which is that it includes a “newline” character at the end of each line—and so the corresponding file.writelines method (which writes a list of lines out to a file) assumes that each line will end with that “now go to the next line” character if necessary. As someone who doesn’t often use Python, that’s an unexpected behavior, so to me this feels like a relatable and very human mistake to make. But anyone used to doing text file operations in Python might find it a strangely elementary thing to miss.

After adding the datetime import, fixing the missing linebreak, changing the script to save to a separate file with an “_edited” suffix, and adjusting the amount of time to shift the captions after some trial and error (not Copilot’s fault), we end up with this as the functioning script:
```
import re
from datetime import timedelta
from datetime import datetime

# Open the SRT file
with open('Three-Body.S01E13.CHINESE.WEBRip.x264-ION10[eztv.re].srt', 'r') as file:
    lines = file.readlines()

# Regular expression to match time codes
time_code_regex = re.compile(r'(\d{2}:\d{2}:\d{2},\d{3}) --> (\d{2}:\d{2}:\d{2},\d{3})')

# Time delta to subtract
time_delta = timedelta(seconds=18)

# Adjust the time codes
for i in range(len(lines)):
    # Check if the line is a time code line
    match = time_code_regex.match(lines[i])
    if match:
        # Check if the caption number is 225 or higher
        caption_number = int(lines[i-1].strip())
        if caption_number >= 225:
            # Parse the start and end times
            start_time = datetime.strptime(match.group(1), '%H:%M:%S,%f')
            end_time = datetime.strptime(match.group(2), '%H:%M:%S,%f')

            # Subtract the time delta
            start_time -= time_delta
            end_time -= time_delta

            # Format the new time code
            new_time_code = f'{start_time.strftime("%H:%M:%S,%f")[:-3]} --> {end_time.strftime("%H:%M:%S,%f")[:-3]}\n'

            # Replace the line with the new time code
            lines[i] = new_time_code

# Write the adjusted lines back to the file
with open('Three-Body.S01E13.CHINESE.WEBRip.x264-ION10[eztv.re]_edited.srt', 'w') as file:
    file.writelines(lines)
```
I saved that as fix_3body.py in the same directory as the srt file, and ran it from a terminal (also in that directory) with python3 fix_3body.py. And that did work—my spouse and I got to finish watching the episode. Hooray! I’m not going to share the edited srt file, but if you’re stuck in the same situation with episode 13, this should get you most of the way to your own copy of a correct-ish subtitle file (I don’t think the offset is exactly 18 seconds, but close enough).

I’ll close with a few scattered thoughts:
- I wonder what caused the discrepency? My best guess is that the show was edited, the subtitles were created, and then approximately 18 seconds of dialog-free footage was removed from the shots of Ye Wenjie being moved from her cell and the establishing shot of the helicopter flying over the snowy forest.
- Overall this felt like a success. As I said, I was not really geared up for programming at the time, and while I could have written my own script to fix this subtitle file, it would have taken a while to get started: what language should I use? How exactly does the SRT format work? How can I subtract 18 seconds from a timecode with fewer than “18” in the seconds position (like 00:16:07,020)? Should I have it accept the file as an input parameter? Maybe have it accept which caption to start with as an input parameter? Should that use the ordinal caption number or a timestamp as the position to start from? Even if I wouldn’t have made the same choices as Copilot for those questions, it got me to something nearly functional without my having to fully wake up my brain.
- Of course, I did have to make at least two changes to make this script functional. Python is very widely used, especially in ML/NLP/AI (lots of text-mangling!) circles, and GitHub Copilot seems to be considered the “smartest” generally-available LLM fine-tuned for a specific purpose like this. Savita Subramanian, Bank of America’s head of US equity strategy, recently asserted on Bloomberg’s Odd Lots podcast (yt) that, “we’ve seen … the need for Python Programming completely evaporate because you can get … AI to write your code for you”. And not to pick on her in particular, but in my experience with AI thus far that’s false, especially the “completely evaporate” part. I’m a computer programmer, so you can discount that conclusion as me understanding in my own interest if you’d like.
- I’m not sure there’s a broader lesson to be taken from this, but it struck me that a broken subtitle file feels like the kind of thing that one should expect when sailing the high seas for content. I was surprised when I saw that even Amazon’s copy of the episode had broken subtitles. In the end, controlling a local copy of the episode and its subtitles allowed me to fix the problem for myself. And in fact this kind of strange problem has been surprisingly common on streaming services. My family ran into missing songs on the We Bare Bears cartoon, and Disney Plus launched with “joke-destroying” aspect ratio changes to The Simpsons, just off the top of my head. I don’t believe I can submit my repaired subtitle file to Amazon, but it seems like there’s some kind of process in place to submit corrections to a resource like OpenSubtitles.org.
intendedeffect

April 18, 2024

Uncategorized

ai, githubcopilot, Programming, python, subtitles
One of the Most University of Chicago Things I Have Ever Read
Well, most humanities at The University of Chicago. This New Yorker profile of Agnes Callard from 2023 is really about one person and her family, but also has so much of what I remember:
1. People determined to be honest, deliberate, and thoughtful about their lives.
2. Children of faculty who are in a very unusual situation, but seem happy.
3. “Night Owls” didn’t exist during my time there, but a late-night conversation series hosted by a philosophy professor sounds like something that I, and many others, would have loved.
4. Minimal acknowledgement that one’s intense self-examination and experience may not be generalizable.
5. Or maybe I mean something that feels like a strange combination of curiosity and insularity?
6. Mental health diagnosis.
7. The strong and often-indulged impulse to philosophical discussion.
8. Horny professors.
9. The kind of restless, striving determination that creates CEOs and political leaders, but applied to something conceptual and niche.
10. An figure who is both a dedicated and recognized theoretician in their academic field, as well as the subject of (what sounds like) a kind of gossipy celebrity culture on campus.
It was not a surprise to learn that Dr Callard was also an undergraduate in Hyde Park (before my time). She sounds like the kind of frustrating and exhausting person to be around that I’ll admit I sometimes miss having in my life.
intendedeffect

March 7, 2024

personal
When do we teach kids to edit?

My 10 year-old recently had to write an essay for an application for a national scholarship program. He did great, writing 450ish words that were readable, thoughtful, and fun, with pleasantly varying sentences. And then we started editing.

I remember feeling like giving up in my early editing experiences, out of sorrow for the perfect first-draft gems being cut and out of bitterness at (generally) my Mom for having the temerity to suggest changes; if my essay wasn’t good enough then she was welcome to write a different one. My kiddo handled it better than I remember dealing with early editing experiences, by thinking through changes with my spouse without losing confidence in his work. But seeing his reactions and remembering my own made me think about how hard it is to get used to editing your writing.

For a kid, and especially one who tends to have an easy time with homework, it’s annoying to have an assignment that feels complete magically turn back into an interminable work in progress as the editing drags on. But that’s the least of editing’s problems! If the first draft was good, then why edit it all—isn’t the fact of the edits an indication that the original was poor work? How can a piece of writing still be mine if I’m incorporating suggestions from other people? Editing for structure and meaning is especially awful, like slowly waking up and realizing that the clarity you were so certain you’d found was just a dream after all.

I’ve seen my kids edit their work in other contexts: reworking lego creations, touching up drawings, or collaboratively improving Minecraft worlds. So maybe there’s something specific to editing writing that’s more challenging—communication is very personal after all. Or perhaps the homework context is the problem—it is a low-intrinsic-motivation setting. Maybe I just think editing is hard because my kid and I were both pretty good at writing, and some combination of genetics and parenting led to us both having a hard time with even the most gently constructive of criticism. There’s certainly some perfectionism being passed down through the generations in one way or another.

My other child is five years old and is, if anything, even more fiercely protective of her creative work. I want to make this better for her, to get her used to working with other people to evaluate and improve something she has made. But how do you do that as a parent? Should I just start pointing out flaws in my kindergartener’s craft projects? (No, I obviously should not.)

Maybe what’s really changing as kids hit their pre teen years is that creative work must increasingly achieve goals that can be evaluated but not measured. Editing starts out as fixing spelling or targeting a word count, but those are comparatively simple tasks. Does your essay convey what you wanted it to? Is it enjoyable to read? Does your drawing represent your subject or a given style as you intended? I would guess that such aims of editing tend to be implicit for highly-educated parents with strong written communication skills, and not at all obvious to an elementary school student. I’m not sure how much groundwork can be laid for these questions in kindergarten, but the next time I’m editing something with one of my kids, I think I’ll try explicitly introducing those goals before getting out the red pen.

intendedeffect

January 9, 2024

Uncategorized

editing, parenting, personal, writing
Was the Cambrian JavaScript Explosion a Zero Interest Rate Phenomenon?

A kind of cliché-but-true complaint common among programmers in the late 2010’s was that there were too many new JavaScript things every week. Frameworks, libraries, and bits and pieces of tooling relentlessly arrived, exploded in popularity, and then receded like waves on a beach. My sense is that this pace has slowed over the last few years, so I wonder: was this, too, a ZIRP phonemon, and thus a victim of rising interest rates along with other bubbly curiosities like NFTs, direct-to-consumer startups, food delivery discounts, and SPACs?

I’ll admit: I can’t say I’m basing this on anything but anecdotes and vibes. But the vibes were strong, and they were bubbly. People complained about the proliferation of JS widgets not because of the raw number of projects, but because there was a sense that you had to learn and adopt the new hotness (all of which still seem to be going concerns!) in order to be a productive, modern, hirable developer. I think such FOMO is a good indicator of a bubble.

And I think other ZIRP-y factors helped sustain that sense that everyone else was participating in the permanent JavaScript revolution. What could interest rates possibly have to do with JavaScript? Let’s say you have $100 to invest in either a startup that will return a modest 2x your investment in 10 years, or a bank account. If interest rates are 7%, your bank account will also have (very nearly) doubled, and of course it’s much less risky than a startup! If interest rates are 2%, your $100 bank account will only end up about $22 richer after a decade, so you might as well take a chance on Magic Leap. It also means that a big company sitting on piles of cash might as well borrow more, even if they don’t particularly know what to do with it. So all of that is to say: low interest rates meant there was cheap money for startups and tech giants alike.

Startups, for their part, are exactly the kind of fresh start where it really does make sense to try something new and exciting for your ~~interactive demo~~ MVP. If you’re a technical founder with enough of a not-invented-here bias—a bias I’ll admit to being guilty of—then you might well survey the JavaScript landscape and decide that what it lacks is your particular take on solving some problem or other. Here’s that XKCD link you knew was coming.

As for the acronymous tech giants, there’s been recent public accusations that larger firms spent much of the 2010s over-hiring to keep talent both stockpiled for future needs and also safely away from competitors (searches aren’t finding me much beyond VC crank complaints, but I’d also gotten this sense pre-pandemic). Everyone loves a good “paid to do nothing” story, but I think most programmers do want to actually contribute at work. A well-intentioned, ambitious, and opinionated programmer might well pick up some slack by refactoring or reimplementing some existing code into a brand new library to share with world. If nothing else, a popular library is a great artifact to present at your performance review.

I started thinking about this in part because it frustrates me how much programming “content” (for lack of a better word) seems to be geared towards either soldiers in corporate programmer armies or individuals who are only just starting their company or their career: processes so hundreds can collaborate, how-to’s fit only for a blank page, or advice for your biannual job change. There’s probably good demographic reasons for this; large firms with many programmers have many programmers reading Hacker News. But something still feels off about this balance of perspectives, and you know what? Maybe it’s all been a bubble, my opinions are correct, and it’s the children who are wrong.

As interest rates proved to be less transient and some software-adjacent trends were thus shown to have been bubbles, a popular complaint was to compare something like crypto currency to the US railroad boom (and bust) of the 19th century—the punchline being that at least the defunct railroads left behind a quarter-million miles of perfectly usable tracks.

So what did we get for the 2010’s JavaScript bubble? Hardware (especially phones), browsers, ECMAScript, HTML, and CSS all gained a tremendous amount of performance, functionality, and complexity over the decade, and for the most part a programmer can now take advantage of all that comparatively easily. The Hobbesian competition of libraries left us with the strongest, and it’s now presumably less complicated trying to decide what to use to format dates.

Then again, abandonware is now blindly pulled into critical software via onion-layered dependency chains, and big companies have taken frontend development from being something approachable by an eager hobbyist armed with a text editor, and remade it in their own image as something to be practiced by continuously-churning replaceable-parts engineers aided by a rigid exoskeleton of frameworks and compilers and processes.

So yes, “in conclusion, web development is a land of contrasts“. To end on a more interesting note, there are two projects that I think could hint at what the next decade will bring. Microsoft’s Blazor is directly marketed as a way to avoid JavaScript altogether by writing frontend code in C# and just compiling it to WebAssembly (yes, that’s not always true because as usual Microsoft gave the same name to four different things). Meanwhile htmx tries to avoid this whole JavaScript mess by adding some special attributes to HTML to enable just enough app-ness on web pages. I would never bet against JavaScript, but it’s noteworthy that both are clearly geared towards people frustrated with contemporary JS development, and it’ll be interesting to see how much of a following they or similar attempts find over the next few years.

intendedeffect

October 6, 2023

Uncategorized
Prevent Unoconv From Exporting Hidden Sheets From Excel Documents
At work we use unoconv—a python script that serves as a nice frontend to the UNO document APIs in LibreOffice/OpenOffice—as part of our pipeline for creating preview PDFs and rasterized thumbnail images for a variety of document formats. Recently we had an issue where our thumbnail for a Excel spreadsheet document inexplicably didn’t match the original document. It turned out that the thumbnail was being created for the first sheet, which didn’t display in Excel because it was marked hidden. I couldn’t find any toggle or filter to prevent unoconv from “printing” hidden sheets to PDF, and ended up having to just add a few lines to unconv itself:
```
# Remove hidden sheets from spreadsheets:
phase = "remove hidden sheets"
try:
  for sheet in document.Sheets:
    if sheet.IsVisible==False:
      document.Sheets.removeByName(sheet.Name)
except AttributeError:
  pass
```
I added that just before the “# Export phase” comment, around line 1110 of the current version of unoconv (0.8.2) as I write this.

I don’t know why this wasn’t the default behavior in LibreOffice, since non-printing export targets like HTML correctly skip the hidden sheet—shouldn’t your printout match what you see on screen?

I’m sharing this as a blog post (rather than submitting a pull request) because unoconv is now “deprecated” according to its GitHub page, which recommends using a rewrite called unoserver in its place. That sounds nice, but for now our unoconv setup is working nicely, especially now that we aren’t creating thumbnails of hidden sheets!
intendedeffect

August 25, 2023

Uncategorized

KnowledgeArchitecture, Programming, Work

A GitHub Copilot AI Anecdote

One of the Most University of Chicago Things I Have Ever Read

When do we teach kids to edit?

Was the Cambrian JavaScript Explosion a Zero Interest Rate Phenomenon?

Prevent Unoconv From Exporting Hidden Sheets From Excel Documents