API For Historical Bus Data

My previous blog post (Visualizing the On Time Performance of Birmingham’s Bus System) looked at creating visualizations based on the real time data provided for Birmingham, Alabama’s public transit system. I have now created a simple, public API that can be used to query the historical data captured from the real time system.

Using my dv8 scripts, I am capturing a snapshot of the real time data every 30 seconds. I am then storing this data in a sql database, so that we will have a historical database. This allows for generating reports and visualizations over time. As of writing this, I have about 2.5 months of data captured.

I am making all of this data public, along with a simple API for making queries. The API is located at:

https://api.dv8.line72.net/

 

However, if you visit that site, you will notice that you get a ‘Page Not Found’ error. That is because it is only a backend API, and you will need to construct URLS to make queries. Let’s look at a few sample queries (All data is returned in JSON format. I have found that making these queries in Firefox is preferred over Chrome, since Firefox will nicely format the results):

Query for all the routes:

https://api.dv8.line72.net/routes

This returned a list of all the known routes and their properties. Each route also has a set of “Trips”. A trip is a unique instance of a bus running along a route. For example, if everyday at 4.00pm a bus leaves Central Station along the route 44 and arrives at its final destination at 5.00pm, this would be a Trip. We can get a list of all the trips for a specific route.

Query for all the trips on the Route 44

Based on the previous query, I have found that the “44 Montclair” has an id of 21.

https://api.dv8.line72.net/route/21/trips

This returned all 61 trips associated with the route 44. Unfortunately, this doesn’t give us that much information about the individual trip. For instance, we have no idea based on the returned results what time of day a trip runs. For every trip, we can get a series of “Waypoints”. Waypoints are all the known information about a bus on a trip at a given time. They are essentially a snapshot, showing the location, passenger count, time deviation, and various other details. We can pick a specific trip and query it.

Query all Waypoints for a single trip on the Route 44

I am going to select a trip at random from the previous query. I will use 699, and will query its waypoints:

https://api.dv8.line72.net/route/21/trip/699/waypoints

This returned A LOT of data. This trip runs every weekday, and we have samples at every 30 seconds. The API returns a maximum of 1000 waypoints, so only the first part of the data was returned. Typically, we want to filter this data, and only get a specific range. For example, let’s only get the waypoints on a specific date. We can use the start_date and end_date parameters to put limits:

Filter the waypoints for a specific date:

https://api.dv8.line72.net/route/21/trip/699/waypoints?start_date=2017-09-05&end_date=2017-09-06

On this date, this trip was an Outbound (0) trip. It left Central Station at 11:45:00 AM and arrived at Eastwood Mall on time at 12:21:00 PM.

We have samples every 30 seconds of this trip. Using that, we could calculated the average on time performance or visualize the number of passengers on the bus. You can take this json data and convert it to CSV using a simple converter tool. You can then import your CSV into a spreadsheet program, like Excel or LibreOffice Calc and create visualizations.

For full documentation on the API including all the routes, parameters, and data types, a Swagger file is available. You can also view the documentation online.

Source Code

Th API was written in PHP using the SLIM framework. The source code is available on my dv8-api-server github project.

Visualizing the On Time Performance of Birmingham’s Bus System

A few months ago, Birmingham, Alabama’s bus system (BJCTA Max Transit) retrofitted all of their buses with a GPS system. Along with this upgrade, they released a website allowing riders and the public to track the locations of buses in real time. The website is backed with a public RESTful API, so I decided I would poll the API to retrieve live information on all the buses throughout the day and store them in a local database.

With this information, it is possible to run some analysis on the system. We could look at passenger count, travel time, travel deviation, areas where buses tend to slow down, performance at different times of the day, and a host of other analytics.

The first analysis I have done is looking at the time deviation. This is how far ahead or behind schedule a bus currently is. A negative number means it is ahead of schedule and a positive number means it is behind schedule. The general rule is that ± 5 minutes is “on time”.

The graph below goes through an entire day looking at all the buses on each route. Each route is separated into its own section, so you can clearly see how a single route performs throughout the day. Along a route, multiple buses will simultaneously be making individual trips along that route. Each color represents a different bus making a different trip. Since there is overlap between buses sometimes making it difficult to see where a trip starts and ends, I have placed bars below the graph. Each of these bars mirrors a single bus making a single trip, allowing you to easily see how many active buses there are and when they started and stopped their trip.

 

If we zoom into a specific route, we can get more detail (Click on the image to see the full day). This is line 17 – Eastwood Mall. It is one of the busier routes with several buses making simultaneous trips. As you can see, most trips start and end on time, however, there is a bit of a peak about halfway through the trips, where the bus is behind schedule by up to 15 minutes.

After looking through the data, I have few interesting conclusions:

  • There is quite a bit of deviation along most routes. However, the most important thing to look at is the start and end of each trip, and for most trips, those have a tendency to converge on 0 deviation. This means the overall trip length is as expected, but buses are being held up in the middle of trips due to traffic and other unforeseeable circumstances.
  • I don’t know what happened to the 14 (Idlewild Palisades) around 5pm. I’m thinking the bus broke down and a new bus (purple) had to be sent out to replace it.

  • Some routes are interlined. For example, the 12 and 18. The bus starts off as the 12, does its trip, but once it pulls into central station, that same bus becomes the 18 and does a trip along route 18. When it returns to central station, it once again becomes the 12. It appears as though the 18 is constantly taking longer than the expected time which makes the 12 start late. Even though the 12 makes up time, it can’t seem to ever get on schedule.

  • The 280 runs down a major suburban corridor that has the worst traffic in the region. Even during rush hour at 5.00pm, the route is expected to have the same trip time as 10am. Clearly it doesn’t.
  • The most heavily used routes: 17, 1, 3, 44, 45, 6, 28, 8, 1 Ex all have minor deviation and run on time.

The code is currently available on github:

https://github.com/line72/dv8

I’m still cleaning up the code, but it is currently usable. I’m also thinking about porting this to D3 and doing live visualization on a website rather than generating a static image.

Global Urban Datafest Hackathon

Last week, myself, Code for Birmingham, and a lot of other talented people came out for the weekend long Global Urban Datafest Hackathon. My team, the Red Mountain COALBOL Miners, worked on a project for the city of Birmingham to help convert their old proprietary datasets into modern formats that they can put in their data warehouse. This is the first big step in getting more open data available for Birmingham.

What was really great about this experience, was every challenge came from a government or city official, so we were solving actual needs and forming great relationships with our city leaders. All five teams did an excellent job, and I hope work continues on all the projects.

My team along with a project working with the Land Bank Authority both won the competition and will be moving on to the nationals. AL.com was one of the sponsors and has done some cool write ups of the weekend:

https://storify.com/AnaMariaCarrano/alabama-smart-cities-hackathon

http://www.al.com/news/birmingham/index.ssf/2015/02/four_teams_compete_to_find_way.html

http://www.al.com/news/birmingham/index.ssf/2015/02/teams_advance_to_global_urban.html

You can find the code our team worked on at our github page.

BJCTA Trip Planner now in sync with official data

My trip planner for the BJCTA Max buses (http://www.bjctatripplanner.org) along with my android app, are now in sync with the official data from the BJCTA. I will continue to keep my site in sync with their data, which means up to date and accurate data!

[appbox googleplay net.line72.bjcta.opentripplanner.android]

Weld Article On The BJCTA Bus System

Weld recently did a two part series on the status of the BJCTA Bus System. My trip planner was featured in the second part. You can read it online from the links below:

Part 1
Part 1

Part 2
Part 2

Part 1 is a very interesting look at the decline of the bus system since the 1950s and the turmoil and political games around transit in Birmingham. Part 2 takes a look at what is being done now to improve the system along with stories from riders who rely on the system.

Sightings of My App

Last night I was waiting on a bus in 5 points after some drinks with friends. Among the 12 of us who were waiting on a bus, I noticed two young girls, who we clearly not from Birmingham, looking at their phones to figure out what bus to ride to get back to their hotel near the convention center. As I looked over their shoulder, I saw they were using my App! That right there was enough to make me realize that all the work I’ve been doing for the last 1.5 years has been worth it! So, if you haven’t already, give the buses a try and use my app to help you.

 

BJCTA Bus Trip Planner

[appbox googleplay net.line72.bjcta.opentripplanner.android]

[appbox appstore 303217144]

 

 

Updated Trip Planner with Route 41 – Fairfield

I have updated the trip planner for the BJCTA Max buses to include new routes. The 41 Fairfield has been completed. You can download the data off of the GTFS Data Exchange.

My Trip Planner already includes the latest data. Or you can download my Android App for free from below.

[appbox googleplay net.line72.bjcta.opentripplanner.android ]

If you have an iPhone, you can download a 3rd party app called Hop Stop.

[appbox appstore 303217144]