Can AI tell us what actually happens in Government?

AI summaries miss key details about government sessions posted online but can identify main ideas. That's still helpful since it is easy to zone out and miss topic shifts.

Author’s Note: I recently completed a class on New York government. Part of the class was attending or watching state government proceedings. As you might expect, watching a State Senate session is incredibly boring. That got me thinking: why not let AI watch it for me?

Google’s Gemini can summarize Youtube videos. In this post I talk about the Senate Committee session I watched and how well Gemini Advanced did to summarize the session. I compare Gemini’s output to the notes I took down manually.

What is the New York Senate?

The New York Senate is one of the two legislative bodies for New York, alongside the Assembly. There are 63 New York state Senators.

What are Standing Committees?

They are permanent committees in the Senate. Proposed bills in the Senate first get funneled into the relevant committee where they get workshopped and then voted on before they can receive a vote in the overall Senate. 

There are 40 standing committees in the New York Senate. Some examples include: 

• Banks

• Budget and Revenue

• Consumer Protection

• Elections 

• New York City Education 

You can see the whole list on this webpage.

What is the Finance Committee’s responsibility?

The Senate Finance committee is best known for collaborating with the Assembly Ways and Means Committee to host public hearings on the state budget and other forms of government spending. 

For example, here’s a press release on Eric Adams testifying to the two committees about city spending. 

Can AI summarize a Finance Committee hearing?

There’s no way the public has the time to actually listen to Senate committee hearings -- this Finance committee session from June 3rd is incredibly boring, poorly produced and lasted almost a whole hour. 

That said, the sessions are posted on Youtube. As artificial intelligence (AI) gets better I suspect people or journalists who want to be more informed about state government could use these tools to inform themselves or help write summaries for others to consume. 

Substack doesn’t format tables very well and I don’t want to bore you with the committee details, so here’s a Google doc on Gemini’s notes on the session compared to my own.

In what ways are AI’s notes better than actually watching the session?

A summarization tool is better than actually watching the video in the sense that it is:

• Orders of magnitude faster: getting key points takes seconds instead of a whole hour. Huge win for viewers.

• Writing notes for you: I often had to pause the video to write down a key point. This meant the process of annotating a video took well over an hour. AI doesn’t need to pause to write down notes.

• Handling poor video/audio quality: Audio and video quality for the Finance Committee session is surprisingly poor, so using a summarization tool avoids having to deal with this. 

  • I would have to rewind the video to understand the video at times due to poor quality. I couldn’t understand the audio when a committee member asked about directed consumer shipping, for example.

• Not going to zone out and miss key details: Some key points in the video are mentioned very quickly, so if you zone out or can’t understand what the speaker is saying, you are out of luck.

  • Example: when the chair notes at the beginning there are no bills, only nominees.

• Defines jargon: Senators use jargon that a lay person wouldn’t grasp, but an AI tool can define these if we prompt it properly.

Where can AI improve?

• Incorrect timestamps: AI missed the end point of discussion on Lily Fan by 12 minutes, which is a pretty large miss in a 49 minute video.

• Doesn’t seem to incorporate visual information: if you watch the video, it’s pretty obvious who Lily Fan is and when she leaves the room at 39:10. The summary doesn’t seem to incorporate this knowledge. 

• Missed the second nominee: for some reason, the AI summary totally neglects any mention of the second nominee, Jon Kamen. This is very strange since he enters and is present in the video for about five minutes — anyone watching would notice Kamen. Also, at the beginning of the session we are told by the Chair that two nominees are present, so the AI did not have this context in mind when writing the meeting summary.

• Missed a lot of interesting details during Q&A: I learned a lot about how the State Liquor Authority (SLA) works and its current issues from the questions committee members asked Lily Fan. The AI summary glosses over juicy details. Stolen bottles of alcohol being sold on the street becomes “enforcement of unlicensed sales of alcohol” in the summary, which is unfortunate. 

• Had to iterate on the right prompting: It was not obvious how to prompt the AI tool to give the details it gave in the summary above. This is a barrier to using the tool productively. Here is the actual chain of prompts I used to arrive at the prompt above: 

  • Can you summarize this Youtube video with bullet points? It will be a long summary: <link>

  • Let's try again -- Can you summarize this Youtube video with bullet points? But also, include details about how the New York Senate works in the summary. Assume I am familiar with U.S. senate but not the New York Senate. It will be a long summary: <link> 

  • Ok your last video summary was almost perfect. But this time, can we also get a sense of how much time (e.g. minutes) was spent on each section of the meeting?

The verdict on using AI? 

Unfortunately, I can’t currently recommend AI as a full substitute for watching the committee session. But, I think journalists and interested viewers should go ahead and use the tool so you know the contours of the session since zoning out and missing topic shifts or key details is inevitable.