One of the concrete benefits of open government data is that third parties can use the data to do something useful that no one in government has the mandate, resources, or insight to do. If you think what I am about to tell you below is cool, and helpful, then you are a supporter of open government data.

On my site GovTrack, you can now find comparisons of the text of H.R. 1, the stimulus bill, at different stages in its legislative life — including the House version (as passed) and the current Senate version (amendment 570).

The main page on GovTrack for HR 1 is: here

Here’s a direct link to the comparison:

Comparisons are possible between any two versions of the bill posted by GPO. Comparisons are available for any bill.

If you find this useful, please take a moment to consider that something like this is possible only when Congress takes data openness seriously. When GPO went online and THOMAS was created in the early 90s, they chose good data formats and access policies (mostly). But the work on open government data didn’t end 15 years ago. As “what’s hot” shifts to video and Twitter, the choices made today are going to impact whether or not these sources of data empower us in the future, whether or not we miss exciting opportunities such as having tools like the one above.

(Thanks to John Wonderlich and Peggy Garvin for some side discussion about this before my post. GovTrack wasn’t initially picking up the latest Senate versions because GPO seems to have gone out of its way to accommodate posting the latest versions before they were passed by the Senate, which is great, but caught GovTrack by surprise.)

There’s a good thread on the mail list raging about the unintended consequences of Members of Congress posting to YouTube to enhance their communications with their constituents. I thought I’d just recap the points here:

It all began when Rob Bluey posted the text of an article that read: “Chester and other critics are asking whether Google [parent company of YouTube] will be able to incorporate data gathered about those who watch the videos for its growing political online advertising business — and whether Congress should be endorsing a for-profit enterprise as a principal access point for official government content. Chester also suggested the YouTube union might create a conflict of interest for those charged with regulating Google and other Internet firms.”

In a perfect world, we could have our cake and eat it too. Opinions differ on what to do in this imperfect world.

David Weller asked if it made sense for Congress to be getting free services from YouTube.

JH Snider expanded on the comparison from C-SPAN in the article. He wrote, “The cable
industry funded C-SPAN to curry favor with Congress and better compete with the local broadcast industry which uses its airwaves in a similar way to curry favor with members of Congress (for details on how this works, see my book: Speak Softly and Carry a Big Stick: How Local TV Broadcasters Exert
Political Power).” He also noted later on that YouTube’s policy could have gatekeeper-type consequences for what the public has access to.

Google, along with other major online services, is known for storing all search queries you make on their site, tracking what websites you visit through their advertising program, etc. I don’t recall Google making any major privacy error, but it’s happened with other organizations. AOL had a major blunder when they published, publicly the search history of a few of their users and essentially didn’t anonymize it. It was a mistake, but mistakes happen. There are data breaches at major off-line companies all the time, leading to fraud and identity theft, so it’s no small question to ask whether constituents should be put in the position of deciding between being civically engaged or protecting their personal information. A Congressional staffer (speaking for himself), unfortunately downplayed the privacy concerns and suggested that if you want to watch the videos anonymously, you can go to a library.

I weighed in and added four more issues to keep in mind: a) We don’t know what kind of access we will have to archival video down the road, b) Reuse of the videos in journalism or education may be hindered now or in the future by copyright, trademark, or terms-of-service limitations enforced by YouTube, and c) YouTube does not currently (but this may change) provide a way for the footage to be downloaded, meaning from a technical standpoint reuse of the video (i.e. journalism/education) is technically not feasible or might violate YouTube’s terms of service), and d) YouTube’s technical choices on how to serve the video may cause legal restrictions on how the videos may be used, owing to third-party software patents.

Greg Elin noted that the laws governing accessibility of websites (i.e. to those who are blind or deaf) might have covered these videos had they been hosted directly on an official congressional website but standards are much lower when congress uses YouTube.

Rob Millis suggested that congressional offices use an existing method that lets the videos be automatically uploaded to YouTube plus other sites. Jad Dedman suggested a second existing site that could be used for this.

In a sense, I offered to create a best-of-both-world service that would make uploading easy(er) for congressional staff and would address the concerns raised by the public. It would, of course, be done at no cost to Congress. I am waiting to be taken up on the offer.

Greg Elin put it very well, and it’s what I personally agree with. What Congress has done so far is major progress, something we have been pushing for some time, and to be applauded. But, just because we (America) took this step doesn’t mean we are done.

UPDATE 1/21: The thread continues under a different subject heading now, as Thomas Lord raises the question of whether any laws were broken by essentially endorsing YouTube.

(email from our google group)

Clay Johnson, Sunlight’s Labs Director, has put together an introductory post and wiki page on how anyone can help participate in the labs work.

In that same spirit, I’ve got three rather simple things that anyone could do to help open the House and Senate.  If editing a wiki is a challenge, don’t worry!  Still-preliminary formatting is among the best of all problems to have, and the editors of Congresspedia are always happy to help, too.

Assemble a clear view of what’s available.  When we first started the Open House Project, I did an inventory of the information House information I could find from official sources.  That wiki page has been cleaned up, and is now available on Congresspedia here.  I started a Senate version, too.  These two pages will help enormously, since we can’t advocate for more openness without fully understanding what’s available.  If anything is missing, please feel free to add it!  The Senate version of the page will have many of the same resources as the House version, but they haven’t been added yet.  We should also consider adding such an information ecology for executive branch data, although I think this work may have been started elsewhere already.  Has it?  Should we start a third (executive) page, and link it to the other two?

Share Events.  If you know of any upcoming events related to transparency in government, add them to our wiki page.  No single news source does a comprehensive job collecting events resources, despite exhorbitant costs, so that’s something that issue advocates should probably take upon themselves to collect.

Share Reform Ideas.  As the Open House Project moves along, and as the Open Senate Project catches up, we’ve got a ton of issues to address.  If something strikes you as under-addressed, or you’ve got a question about how to find some piece of information or data, please ask.  I prefer that we err on the side of too much (relevant) email, rather than not enough.

thanks!

John

Here’s a New Year’s Resolution I wish the Senate would adpot: We will be painfully serious about using technology to support transparency.

Now, the Senate does some things right: They have a new lobbying disclosure database. They make their membership, member contact info, and committee schedule information available to the public as XML, which helps others make the information more readily accessible to the public. I hear that we will see some exciting developments in terms of (Member-to-public) constituent communication very soon. So I know that the Senate has it in them to really take things to the next level.

Technology is a key component of transparency. It’s the only way the public can hope to navigate the complexity of government. And, fortunately, this kind of technology is pretty cheap. We don’t need to build space shuttles or create new cabinet-level departments to get radically improved transparency. That’s what always fascinates me about this. It’s so damn EASY to make huge progress here, but few want to take the first steps.

In 2009, I hope to see staffers in the Senate talk to their bosses about ways to capitalize on technology to radically improve the public’s ability to see what’s going on down there in D.C. We, the Open Senate Project, are going to give you some ideas, just as soon as we get to writing our report.

(cross posted from our Google Group)

Jon Henke wrote the following provocation, and I decided to respond to the whole list, since it’s a topic I think many here will be interested in.  (I asked his permission to post in full.)

His email is first, and then my responses:

(and now responses)

I agree with the first sentence.  There’s a backlog of reports requested, though, and first priority is given to committee chairs’ requests.  (Here’s the GAO document (pdf) that lays out their triage arrangement for requests.)  They’re also all public, although some of them are withheld from web publication by agency request.

I’m not sure how likely it is that GAO (or Congress) would prepare a report on transparency; CRS is more likely to respond (or to be asked) to prepare such a report.  One recent, very similar report was prepared by Walter Oleszek, which I reviewed here, in September of 2007.  I’m not sure if the report has been updated since, but it’s a great resource.

Applying GAO to Congress isn’t done all that frequently (as far as I’m aware), since GAO is a legislative support agency (like CRS or CBO), though of a slightly different type, acting often more like an executive agency.  GAO’s statutory requirement to review financial disclosure within all 3 branches has been (apparently) routinely ignored since approximately 1990.

With those cautions in mind, I’d love to see a GAO review of transparency in Congress, or even across all three branches, especially since many of the disclosure requirements have roots in the same laws (like the Ethics in Government Act, which sets financial disclosure requirements for high ranking staff in all three branches.)  This is probably one among many tools to spur Congress into some constructive introspection, including CRS reports, committee hearings, legislation, sense of the Congress resolutions, informal advisory recommendations (cf this list, OMBWatch’s report, or the ABA rulemaking report), lawsuits, committee reports, rulebreakers and internal champions, media coverage, etc.   It would likely have to come from some committee chair (or leadership figure) with relevant jurisdiction, like a leg. branch approps chair, government reform chair (in the House) or gov’t affairs cmte chair (in the Senate).

I agree with this in spirit, but I strongly fear unintended consequences here.  Take the Office of Legislative Counsel, for example.  This is a quiet, isolated room full of legal scholars with expertise in the actual drafting of laws.  They operate under strict guidelines over what they can share, since the same drafters could conceivably be asked to write opposing bills for political opponents.  Maintaining confidentiality and objectivity is essential, since everyone has an interest in trusting that bills will be well written.

The same is true for some CRS documents (those we’re NOT asking to have published), where Members sometimes turn for confidential advice.  I think confusion between these documents and the circulable documents causes some of the resistance to an open CRS we find on the hill.

GAO, not unlike a police department, needs whatever tools are necessary for effective oversight, and publishing only completed documents probably falls into that category.

Again, with those cautions in place, I’m a HUGE fan of the idea that an empowered public can strengthen our checks and balances, and reinstate effective oversight both within and between branches.  That’s why I’d like to see a collection of upcoming reports to Congress (since it would empower public watchdogs and oversight staff alike), and why I started the (now defunct) Congressional Committees Project two years ago.

Integrating public activity into carefully constructed government processes can be tricky, which is one reason that the Peer to Patent Project’s success is so impressive — it’s obviously the product of some brilliant and careful planning.

I’d love to see oversight committees, GAO, or even CRS start to address the possibilities presented to them by communities such as this one, but the first steps are hard to find, especially for as heated a topic as oversight, where jurisdictions are carefully protected, and where the stakes are especially high for those under investigation.  If even commenting on the quintessentially public bill is so hard to implement, a publicly empowered oversight process will be even tougher to generate.

It’s a good first step, though, that people within those institutions we wish to change are following along closely (many are on this list), and that policy makers and the media are recognizing the developing capacity for public relevance.

(cross posted from OHP)
Many acts of Congress are passed with a reporting requirement. Throughout the US Code are various requirements that agencies submit reports to Congress. This is a good thing, since strong congressional oversight can create constructive incentives within agencies (and, by extension, the entities those agencies regulate).

Here’s the problem: agencies often ignore these reporting requirements entirely, turn in reports late, or don’t even know what reports are required of them. Staff and Members on oversight committees are sometimes similarly delinquent, unaware of what reports may be pending or overdue.

Anytime the overseers and the overseen aren’t aware of the reporting they’re required to prepare, oversight has no chance of doing its job. If this is sometimes the case within the executive and Congress, what chance does the public have of keeping track of what government is doing? Pretty slim.

There’s no centralized list of upcoming reports, so congressional oversight staff, journalists, and agency staff have no menu of reporting responsibilities, and are stuck making assumptions based on what is often an incomplete view of statutorily mandated responsibilities.

I’m proposing a solution to this situation, see the end of this post for a summary.

A few months ago, I suggested a partial fix to this problem — a requirement that oversight plans, as prepared by congressional committees in the US House (and not the Senate) be given two additional requirements. First, I suggested that after the oversight plans for each coming Congress are created, they should be made public by the Clerk of the House and the Oversight and Government Reform Committees, where they’re only required to be submitted now, not published. (The House Rules Committee, for example, does this now. I’d like to see a page or document with all committees’ oversight plans.) This would provide a central place to see the collected oversight plans from across all House committees, not unlike the document now compiled that has all House committees’ rules. That would give a centralized congressional plan for oversight, across committees, but wouldn’t ensure that committee staff, agency staff, or the public knew what reporting to expect from the agencies.

That’s the second part of my earlier suggestion: to require that House committees’ oversight plans contain a list of expected reporting within their jurisdiction. (I’m ignoring the Senate here.) It turns out that this requirement is suggested within the House Oversight Manual, under the section “Selected Oversight Techniques:”

1. Prepare a document, as needed, which outlines for each subcommittee of a standing committee the agencies, laws, programs activities, functions, advisory committees, and required agency reports that fall within its jurisdictional purview.

So that’s where my proposal stood a week ago: amend House Rules to require a list of expected reports from agencies, and provide a central place from which to access all of the oversight plans.

I still like that proposal, but further reflection reveals this as utterly insufficient. How can effective oversight be mandated through self-imposed rules changes on Congress? I still hold that those two changes would go a long way toward encouraging oversight, but without addressing the negligence or disorganization within the executive, Rules changes can only help so much.

After several conversations with subject matter experts, I have two additional suggestions for strengthening Congressional (and public) oversight of the Executive Branch.

First, the first step toward any legislative or rules changes will probably be to appreciate the scope of the reporting requirements. CRS should be instructed to do a review of all of the pending agency report expected over, say, the next year. If this were organized both chronologically and by agency, that document would immediately be immensely valuable to all committee staff preparing the next Congress’s oversight plan, and also to the agency staff responsible for prepring the reports. It would tell them what’s expected from them, and what they can expect to have delivered to them, and when.

This CRS review of pending agency reports should also prove the value of having an aggregated reporting list to agency officials and Members of Congress both, regardless of political affiliation. If you’re inclined to be critical of the next administration, then a menu of report cards should look like opportunity for criticism. If you’re inclined to defend the next administration, then accountable and effective agencies should know when their next reports to Congress and the public are due.

Second, ongoing publication of expected (and overdue) reports (by chronology and agency) should be a responsibility taken on by the administration. Two candidates for this responsibility:

OIRA - The Office of Information and Regulatory Affairs OIRA, part of OMB, seems like a natural fit, since congressional reporting is agency-created information. OMB seems wary of taking on enforcement responsibility, so this could be a hard sell.

OLC - The Office of Legal Counsel OLC, part of the Department of Justice, is responsible for interpreting the law on behalf of the administration. Who better to remind the agencies of just what their legally mandated responsibilities are?

I’d just like to add that there’s some precedent for this sort of thing, in the Unified Agenda, or “Semiannual Regulatory Agenda” which gives a prospective view of the next six months of rulemaking activity.

A “Semiannual Reporting Prospectus” should be similarly compliled, submitted to the Congress itself, and published in the Federal register.

If financial reports, political polls, annual job reviews, and students’ report cards can all be organized and coordinated so that we live in expectation of what they’ll say, shouldn’t we be able to do as well with our government? The US Code may be as splintered and complex as the committee jurisdictions that oversee them, but our public view of their performance doesn’t have to suffer from their inherited discord.

A few simple adjustments can clarify and strengthen oversight of executive agencies:

• Committees should write oversight plans, which include what reports they expect to see from agencies, with dates (as required by law).
• Congress should publish these reports in a central document or location.
• Congress should request a CRS survey of all required agency reports for the coming year.
• The executive branch should assign responsibility for oversight reporting to a specific agency, and submit a menu of upcoming reports to Congress and publish it in the Federal Register.

Additional info:

Here’s a few reporting requirements I picked out of the US Code:

strategic plans

responsibilities of the Secretary of Agriculture

and here is a reporting requirement of the Secretary of Education

Sunlight has just sent out the following press release, announcing the launch of the Open Senate Project.

As Ellen blogged this morning, we’re delighted to have Majority Leader Reid’s endorsement of the effort, so we can apply the same public discussion and analysis to Senate reform priorities as we’ve applied to the House.

We have created a separate blog site to track discussion and developments, and to collect resources, but we’ll be using the same google group for Senate discussion, since duplicating the collection of people following this list would be impossible, and much of the discussion here already applies to both chambers (and often the executive).

I’m also happy to announce that Jon Henke and Josh Tauberer have agreed to help coordinate the discussion and preparation of the recommendations report; the effort will certainly benefit from the expertise they can offer.

Finally, I’d like to offer my gratitude to everyone for contributing and following along as we work our way through Congress’s public face — this project’s strength comes from the (sometimes heated) dialog and debate on this list.  This email list is all the evidence we need that public interaction can bring substance and weight to even the most obscure or administrative of policy issues, and that a shared stake in policy outcomes can coalesce disparate communities into something forceful.

Thanks also to Majority Leader Harry Reid, who has acknowledged the value of public dialog by taking a proactive step in welcoming a conversation outside the comfortable boundaries of political party, the institutions of Congress, and the world of familiar advisers and trusted staff.  Even though nothing in the report is binding in any way, Reid’s embrace of such an open forum will help encourage technological advances, and demonstrate how even a traditional and collegial institution such as the US Senate can take advantage of new communities forming online.

I’m looking forward to working with Senate leaders and staff from both parties, and kicking off the conversation soon about what the Senate might do differently.

Here’s an embedded copy of the Senate Rules for the 110th Congress, for reference. (Taken from the Senate Rules Committee, here.)

Get your own - Open publication

One of the unique things that the Open House Project has brought to Congress is the perspective of the public internet user.  Since, until recently, there was no Internet, we’re in a situation where a very traditional and complex institution is seeking to adjust to very new pressures and expectations.

The public that can go online as empowered consumers, researchers, and members of new communities — that same public expects Congress to be especially available, since legislative information places a much more fundamental role than, say, lottery numbers or restaurant menus.  Our expectations are growing.

This was obvious during the recent bailout/financial rescue situation, when leaders in the House posted the bill online, and public demand shut down the servers, which couldn’t handle the interest. (Even the copy I posted now stands over 31,000 views.)

Congress has some work to do, in keeping up with expectations online.  Sometimes quite literally.

The Open Senate Project is designed to help the Senate adjust to those expectations.

Everyone has a stake in congressional information, and some people are well situated to help articulate what their needs are.  Bloggers, journalists, activists, open government advocates, congressional staff, Members of Congress, newspaper readers, researchers, and even both political parties — these perspectives all have something valuable to offer Senate leaders, who have the difficult job of shaping a very complex institution into a 21st Century mold.

Those constituent groups, however, haven’t always worked together effectively before.  They operate through very different power structures, which are often inefficient, and have goals that shape their priorities.

A public dialog has the power to overcome these shortfalls — to incubate new ideas, and to connect the work of those who share a stake in any given reform.

This has been the strength of the Open House Project, where a community of interest has grown around complex issues of, congressional authority, IT management, grassroots involvement, and institutional organizing.

Welcome to the Open Senate Project!

We’re excited to get things rolling here, and to help the Senate take full advantage of developing technology to enable citizens to work more closely with their Senators, and to understand their work.

We’ll still be blogging on the Open House Project blog, where you can see hundreds of posts about transparency reform in the House of Representatives.

Also, the bulk of the public work on the Open Senate Project will happen on the Open House Project google group, which has now grown (at the time of writing) to almost 400 members.

As we get things started, we’ll be explaining what to expect, and seeding the discussion about what the Senate could do differently to empower the constituents they represent.

If you’re new to the project, in addition to the Open House Project, check out the Sunlight Foundation, the organization responsible for this work, and also, if you’d like to start digging, here are the Rules of the Senate.

In case you’re stumbling across this before Tuesday, we’ll have an official announcement then, so stay tuned!