RTPI backend system to be updated

CatInABox

The NTA have announced that they're moving their bus related RTPI API to GTFS-R. This should make RTPI more relevant and easier to use. More details should also start appearing in places like Google Maps, etc.

https://twitter.com/TFIupdates/status/1293162864518352896

They're phasing the old one out pretty damn quick as well, end of August there'll be a temporary shutdown, and by the end of September it'll be permanently gone.

Eduard Khil

Really hope it's actual GPS location position on board the buses to the point of pinging the actual location of the bus in the case of Dublin Bus in particular it might help with the buses going missing from the time schedule and may provide a more accurate information system.

ITV2

Eduard Khil wrote: »

Really hope it's actual GPS location position on board the buses to the point of pinging the actual location of the bus in the case of Dublin Bus in particular it might help with the buses going missing from the time schedule and may provide a more accurate information system.

Once of course the bus logs on correctly, sometimes the TGX machine fails to log on. (like for me this morning).

VG31

Eduard Khil wrote: »

Really hope it's actual GPS location position on board the buses to the point of pinging the actual location of the bus in the case of Dublin Bus in particular it might help with the buses going missing from the time schedule and may provide a more accurate information system.

They already have the GPS locations for Dublin Bus in the NTA Real Time App. The UI isn't great though. The GPS information isn't public otherwise I'd probably make a nicer looking version myself.

Hopefully this new API includes GPS locations.

Eduard Khil

Addendum to my post I hope there is a report button/log for buses that go out of service or take an extended break RTPI and NTA apparently fine both Dublin Bus and Go Ahead Ireland for poor delivery of certain services the passengers in Finglas particularly are very affected by the 40, 40D 140 and 17a I'd love a way to report buses not turning up that will actually be seen by the powers that be.

Capwell

Now that the NTA GTFS Realtime feed has supposedly gone live, has anyone had a look at it. I've registered for the API key and spent the last few days building a test feed in Python. The data seems to be incomplete and / or wildly inaccurate. For example my local bus stops in Dublin don't appear at all. I'd be interested in anyone else's experience.

harringtonp

See this thread guys

https://www.boards.ie/vbulletin/showthread.php?t=2058122471

or just try out https://yourstop.info

smackyB

Capwell wrote: »

Now that the NTA GTFS Realtime feed has supposedly gone live, has anyone had a look at it. I've registered for the API key and spent the last few days building a test feed in Python. The data seems to be incomplete and / or wildly inaccurate. For example my local bus stops in Dublin don't appear at all. I'd be interested in anyone else's experience.

Not sure if you know but it's meant to be used in conjunction with the GTFS file - the GTFS-R feed just tells you about trips in progress and how to adjust the timings for those trips in the GTFS file.

I've found it patchy so far, had quite a few reports from users of my app saying the times are off when they compare it to the official Dublin Bus app. From looking at the raw data I've seen quite a few cases where Dublin Bus trips don't show up in the GTFS-R file but are present in the Dublin Bus app API response. The GTFS-R seems reliable for GoAhead routes though.

Use the apisupport@nationaltransport.ie address for issues you find, the more people that are reporting them the more likely they are to do something about it.

harringtonp

smackyB wrote: »

Not sure if you know but it's meant to be used in conjunction with the GTFS file - the GTFS-R feed just tells you about trips in progress and how to adjust the timings for those trips in the GTFS file.

I've found it patchy so far, had quite a few reports from users of my app saying the times are off when they compare it to the official Dublin Bus app. From looking at the raw data I've seen quite a few cases where Dublin Bus trips don't show up in the GTFS-R file but are present in the Dublin Bus app API response. The GTFS-R seems reliable for GoAhead routes though.

Use the apisupport@nationaltransport.ie address for issues you find, the more people that are reporting them the more likely they are to do something about it.

Yes there is good RTD coverage for Dublin bus during the days but then once you get to early evening the number of routes covered drop way off. The other 3 operators covered don't show this trend. I've asked about this, got no response yet but I get the impression it is an improving situation.

I suspect there own app is not reading directly from the feed but somewhere earlier in the pipeline.

On a different note I noticed that route numbers can sometimes be different in their published data to what the app shows. For example on the Blanchardstown to Tallaght route their app lists a 76a but in the data it shows as 761. Anyone know what is going on here, has there been a recent route name change ?

smackyB

harringtonp wrote: »

Yes there is good RTD coverage for Dublin bus during the days but then once you get to early evening the number of routes covered drop way off. The other 3 operators covered don't show this trend. I've asked about this, got no response yet but I get the impression it is an improving situation.

I suspect there own app is not reading directly from the feed but somewhere earlier in the pipeline.

On a different note I noticed that route numbers can sometimes be different in their published data to what the app shows. For example on the Blanchardstown to Tallaght route their app lists a 76a but in the data it shows as 761. Anyone know what is going on here, has there been a recent route name change ?

Yeah I've asked them to fix this multiple times, GoAhead routes with letters are erroneously named with a corresponding number of the alphabet instead. I've had to manually fix this myself in the GTFS file.

Good to know you've seen similar regarding patchy Dublin Bus data. It's frustrating TFI's own apps don't use the same datasource, otherwise I bet they would be much more proactive about fixing this.

harringtonp

I draw the line myself at doctoring GTFS data.

The quality of GTFS headsign data has forced me to improve the implementation though. With Bus Eireann in particular, the headsign often contains the route name (e.g. Ennis Bus Station - Galway Bus Station) and sometimes buses stop short of the destination which can be misleading. So when appropriate I show the first and last stop name too.

The TFI app loads data outside of GTFS when displaying stop names so you get for example the area along with the road name. They do make this available (Naptan, NPTG) for consumption but you would be working outside of GTFS and the implementation would only be good for TFI. In a good GTFS implementation (if anyone from TFI is reading this...) the stops.txt file would have everything you need. They only use stop_name but ideally would use the stop_code and stop_desc fields too.

Pity they don't make the vehicle position feed available. There's not much that can go wrong with this as a timestamp and lat/lng are provided. And as long as users can see where the bus is, wayward predictions don't matter so much.

Victor

harringtonp wrote: »

On a different note I noticed that route numbers can sometimes be different in their published data to what the app shows. For example on the Blanchardstown to Tallaght route their app lists a 76a but in the data it shows as 761. Anyone know what is going on here, has there been a recent route name change ?

On the timetable sheets, these routes have been renumbered.

smackyB

Victor wrote: »

On the timetable sheets, these routes have been renumbered.

No sign of that on GoAhead's website, where have you seen that?

smackyB

Added an update to my apps to show which buses are being tracked in the GTFS-R system (grey is a non-realtime bus). Very poor coverage this morning, only 3/17 results are real-time

harringtonp

None at the moment realtime showing on mine

But it seems to fall out most evenings

harringtonp

Where as Go Ahead is fine

Capwell

Thanks for your reply. I assumed it might work similarly to the old one. The GTFS data spec has the ability to include the arrival time but I guess it depends on the transport authority's implementation. I had the old one integrated into my home automation system to display it on a wall mounted screen (very popular with the family). So I thought I could just drop in the new version, but not so it seems. I have done a good bit of work to build a new version using all the GTFS static data which wasn't needed in the old version.

Does anyone have a full decode of the Service IDs, e.g. "2#1" looks like it is Mon to Fri. Dublin Bus was using "3#1" for the bank holiday. Thanks

Victor

Capwell wrote: »

Does anyone have a full decode of the Service IDs, e.g. "2#1" looks like it is Mon to Fri. Dublin Bus was using "3#1" for the bank holiday. Thanks

If this is the service profile (how soon it gets to each stop), it seems there are no less than 17 of them for different types of days and times of day.

harringtonp

Anyone seeing a lot of read time outs when polling the RTD URL ?


WARNING [2020-10-28 09:58:10] Error reloading Trip Update https://gtfsr.transportforireland.ie/v1/ [Read timed out]
WARNING [2020-10-28 09:58:45] Error reloading Trip Update https://gtfsr.transportforireland.ie/v1/ [Read timed out]
WARNING [2020-10-28 10:00:27] Error reloading Trip Update https://gtfsr.transportforireland.ie/v1/ [Read timed out]
WARNING [2020-10-28 10:02:46] Error reloading Trip Update https://gtfsr.transportforireland.ie/v1/ [Read timed out]
WARNING [2020-10-28 10:03:21] Error reloading Trip Update https://gtfsr.transportforireland.ie/v1/ [Read timed out]
WARNING [2020-10-28 10:08:10] Error reloading Trip Update https://gtfsr.transportforireland.ie/v1/ [Read timed out]
WARNING [2020-10-28 10:08:45] Error reloading Trip Update https://gtfsr.transportforireland.ie/v1/ [Read timed out]
WARNING [2020-10-28 10:12:11] Error reloading Trip Update https://gtfsr.transportforireland.ie/v1/ [Read timed out]
WARNING [2020-10-28 10:13:19] Error reloading Trip Update https://gtfsr.transportforireland.ie/v1/ [Read timed out]
WARNING [2020-10-28 10:16:35] Error reloading Trip Update https://gtfsr.transportforireland.ie/v1/ [Read timed out]

Capwell

There seems to be no obvious linkage between the agency_id and the service_id or am I missing something. There seems to be no rule for selecting the service_id to be used for selecting the trips and then the stop_times. Dublin bus seems to use 1 set of IDs, e.g. 2#1, 1#1, 3#1 and Goahead is using a different set, e.g.10706, 10707 etc.

Does anyone know the rules?

harringtonp

Capwell wrote: »

There seems to be no obvious linkage between the agency_id and the service_id or am I missing something. There seems to be no rule for selecting the service_id to be used for selecting the trips and then the stop_times. Dublin bus seems to use 1 set of IDs, e.g. 2#1, 1#1, 3#1 and Goahead is using a different set, e.g.10706, 10707 etc.

Does anyone know the rules?

It's a lot more work than a straight forward api.

Read the spec carefully at

https://developers.google.com/transit/gtfs/reference

For your purposes focus on the agency, routes, trip, calendar, stops and stop times tables.

The information is spread across all these files and if you're loading them into equivalent db tables you'll be constructing a sql query across them

harringtonp

The GTFS RT Api has been down for me since 3pm yesterday. The call succeeds but there is no data.

Anyone else seeing data ?

Capwell

Hi, thanks for your reply. I have been able to build a version which works fine for me (still testing) but I still have that one issue about the service_id. A quick scan of the spec doesn't show which service_id to use for a specific agency, i.e. Dublin Bus or Go Ahead. My 2 local stops are 1 of each. So for now, I have hard wired the linkage based on:

Dublin Bus
Mon-Fri "2#1"
Sat "1#1"
Sun "3#1"

Go Ahead
Mon 10701
Tue 10702
Wed 10703
Thu 10704
Fri 10705
Sat 10706
Sun 10707

I still don't have a solution for Bank Holidays, Easter, Christmas but I'll have a look at the spec in more detail later. The TFI app was not showing real time data yesterday and again this morning.

harringtonp

Hi,

In trips.txt you have service_id and route_id along with other trip information

In routes.txt you have route_id and agency_id along with other route information

And in agency.txt you agency_id and agency_name (and other agency information)

So you derive the agency name from the service id via an effective join of these 3 files.

calendar_dates can remove specific days for a service and add days such as bank holidays to an existing service or a brand new service.

Capwell

Thanks for the reply, it's given me an idea how to solve it based on the way I built it for my system, or later on, I may try a different approach.

harringtonp

It can be fun figuring this out but if you're looking for an easier way you're welcome to leverage of my backend. Send me the stop ids of the stops you are interested in and I will construct the URL(s) for you. Data is returned in JSON format

smackyB

harringtonp wrote: »

The GTFS RT Api has been down for me since 3pm yesterday. The call succeeds but there is no data.

Anyone else seeing data ?

Same here

Capwell

harringtonp wrote: »

It can be fun figuring this out but if you're looking for an easier way you're welcome to leverage of my backend. Send me the stop ids of the stops you are interested in and I will construct the URL(s) for you. Data is returned in JSON format

Hi, thanks for the offer but I have been able to work it out. I have built a 'bus sensor' which is integrated into my home automation system (Home Assistant running on a Raspberry Pi3), replicating the setup I had with the previous API. It will require more maintenance because of all the static reference data required and the Calendar information which will need to be refreshed at year end I suppose.

Capwell

The new data for part of 2021 has been issued on the NTA web site. A lot of the route IDs have changed, e.g.
route:

'60-16-d12-1'
is now
'60-16-b12-1'

anybody know why?

harringtonp

I don't but can tell you they were having problems with route ids. Sometimes, when unscheduled trips are added in GTFS RT they don't reference an existing route. Other times, when you see two added RT entries for a single trip the only difference between them is the route id. One of the route id's will reference routes.txt and the other won't. I highlighted this and unlike other issues, a case number was issued. If this is a side effect of a solution then great, I will have to check it out at some point.

harringtonp

Actually just looked back over examples I sent in and route ids which didn't reference routes.txt had d12 in them. Those which did had b12

Are you taking your information solely from tripUpdates ? If so I'd double check that any route ids you use exist in routes.txt. To conform to the spec it should.

Capwell

In my home solution I do start with tripUpdates and work from there. I have a bit of config in Home Assistant which specifies the Routes and Stops I am interested in, i.e. those near my house. The solution stopped working after I updated to the new static data and the issue was the change in Route IDs. Once I changed these in my config, everything works fine again. I was just wondering why they changed them.

harringtonp

Who knows why the route_id changed. While it is GTFS best practice to persist it across feed iterations it is not a requirement of the spec.

You might find the route short name works better but there would be an extra level of indirection. Basically while you may get away with TripUpdates as a starting point, the design of GTFS is that it supplements the static feeds.

smackyB

Looks like realtime coverage has disappeared again for Dublin Bus. GoAhead seems fine. There must be something very unreliable that bridges between Dublin Bus' realtime system and the NTA's GTFS system as DB journeys have been intermittently missing from the GTFS-R system for months now. Very frustrating :mad:

Stevek101

Has the RTPI.ie service been discontinued?

p_haugh

Stevek101 wrote: »

Has the RTPI.ie service been discontinued?

It is still working fine for me currently.

Victor

It's working now.

Stevek101

It use to show real time now it seems to only show predicted. It's been like this since the update in early December.

p_haugh

Stevek101 wrote: »

It use to show real time now it seems to only show predicted. It's been like this since the update in early December.

Now that I double check it, it's showing predicted times for Dublin Bus routes - go ahead, LUAS and Bus Eireann are still showing real-time however.

Capwell

harringtonp wrote: »

It can be fun figuring this out but if you're looking for an easier way you're welcome to leverage of my backend. Send me the stop ids of the stops you are interested in and I will construct the URL(s) for you. Data is returned in JSON format

harringtonp wrote: »

Who knows why the route_id changed. While it is GTFS best practice to persist it across feed iterations it is not a requirement of the spec.

You might find the route short name works better but there would be an extra level of indirection. Basically while you may get away with TripUpdates as a starting point, the design of GTFS is that it supplements the static feeds.

Yes, I replied too quickly last time. I start with Trips and find the next trips passing my local stops for today and then eliminate the ones I am not interested in before applying the realtime data to the relevant stops on the trips selected. During the early part of the pandemic, there was often no stop time update as I guess the buses were running on time as per the schedule. This is probably also true over the holidays.

Capwell

if anyone is interested in how it fits into Home Assistant (as a 'bus' sensor) follow the link below where I posted the code. I use it to drive a permanent display in my kitchen for my 2 local buses. Uses a Nextion LCD display.

https://community.home-assistant.io/t/gtfs-and-or-gtfs-realtime-not-working/147761/17?u=jerry_or

p_haugh

Not sure if this was recent or not, but the updated gtfs realtime feed has now been fully integrated into Google & Apple maps

Stevek101

p_haugh wrote: »

Not sure if this was recent or not, but the updated gtfs realtime feed has now been fully integrated into Google & Apple maps

Times are completely different to DB. Real time never recovered from it going down before Christmas. Only the DB seems to be in any way accurate.

harringtonp

Stevek101 wrote: »

Times are completely different to DB. Real time never recovered from it going down before Christmas. Only the DB seems to be in any way accurate.

Apologies but what is "the DB" ?

Stevek101

harringtonp wrote: »

Apologies but what is "the DB" ?

No bother at all, it's Dublin Bus.

harringtonp

Anyone else find the GTFS realtime feed has no data ? From about 4am this morning

smackyB

harringtonp wrote: »

Anyone else find the GTFS realtime feed has no data ? From about 4am this morning

Yep down for me too. Have sent an email to NTA but no reply. Suspect it won’t be fixed until next week.

GT89

Stevek101 wrote: »

No bother at all, it's Dublin Bus.

Could be Deutsche Bahn I suppose

harringtonp

GT89 wrote: »

Could be Deutsche Bahn I suppose

Only thing I could see in my head at the time was DataBase. That's what you get from years of working with them ðŸ˜

smackyB

Reply from NTA just now:

Hi,

We are currently investigating the upstream system to identify the root cause

regards

NTA

EDIT: looks like its back up now

r93kaey5p2izun

I just get 404 error on rtpi now. Also DB has not been actually live tracking since Christmas on any route I take anyway. Go Ahead is live but DB is just giving the timetabled arrival time regardless of reality. Surely that is utterly unacceptable at this stage?!