Advertisement
If you have a new account but are having problems posting or verifying your account, please email us on hello@boards.ie for help. Thanks :)
Hello all! Please ensure that you are posting a new thread or question in the appropriate forum. The Feedback forum is overwhelmed with questions that are having to be moved elsewhere. If you need help to verify your account contact hello@boards.ie

RTPI backend system to be updated

  • 11-08-2020 2:02pm
    #1
    Moderators, Computer Games Moderators, Recreation & Hobbies Moderators Posts: 10,193 Mod ✭✭✭✭


    The NTA have announced that they're moving their bus related RTPI API to GTFS-R. This should make RTPI more relevant and easier to use. More details should also start appearing in places like Google Maps, etc.

    https://twitter.com/TFIupdates/status/1293162864518352896

    They're phasing the old one out pretty damn quick as well, end of August there'll be a temporary shutdown, and by the end of September it'll be permanently gone.


«134

Comments

  • Registered Users Posts: 796 ✭✭✭Eduard Khil


    Really hope it's actual GPS location position on board the buses to the point of pinging the actual location of the bus in the case of Dublin Bus in particular it might help with the buses going missing from the time schedule and may provide a more accurate information system.


  • Registered Users Posts: 449 ✭✭ITV2


    Really hope it's actual GPS location position on board the buses to the point of pinging the actual location of the bus in the case of Dublin Bus in particular it might help with the buses going missing from the time schedule and may provide a more accurate information system.

    Once of course the bus logs on correctly, sometimes the TGX machine fails to log on. (like for me this morning).


  • Registered Users Posts: 3,379 ✭✭✭VG31


    Really hope it's actual GPS location position on board the buses to the point of pinging the actual location of the bus in the case of Dublin Bus in particular it might help with the buses going missing from the time schedule and may provide a more accurate information system.

    They already have the GPS locations for Dublin Bus in the NTA Real Time App. The UI isn't great though. The GPS information isn't public otherwise I'd probably make a nicer looking version myself.

    Hopefully this new API includes GPS locations.


  • Registered Users Posts: 796 ✭✭✭Eduard Khil


    Addendum to my post I hope there is a report button/log for buses that go out of service or take an extended break RTPI and NTA apparently fine both Dublin Bus and Go Ahead Ireland for poor delivery of certain services the passengers in Finglas particularly are very affected by the 40, 40D 140 and 17a I'd love a way to report buses not turning up that will actually be seen by the powers that be.


  • Registered Users Posts: 20 Capwell


    Now that the NTA GTFS Realtime feed has supposedly gone live, has anyone had a look at it. I've registered for the API key and spent the last few days building a test feed in Python. The data seems to be incomplete and / or wildly inaccurate. For example my local bus stops in Dublin don't appear at all. I'd be interested in anyone else's experience.


  • Advertisement
  • Registered Users Posts: 2,562 ✭✭✭harringtonp




  • Registered Users Posts: 746 ✭✭✭smackyB


    Capwell wrote: »
    Now that the NTA GTFS Realtime feed has supposedly gone live, has anyone had a look at it. I've registered for the API key and spent the last few days building a test feed in Python. The data seems to be incomplete and / or wildly inaccurate. For example my local bus stops in Dublin don't appear at all. I'd be interested in anyone else's experience.

    Not sure if you know but it's meant to be used in conjunction with the GTFS file - the GTFS-R feed just tells you about trips in progress and how to adjust the timings for those trips in the GTFS file.

    I've found it patchy so far, had quite a few reports from users of my app saying the times are off when they compare it to the official Dublin Bus app. From looking at the raw data I've seen quite a few cases where Dublin Bus trips don't show up in the GTFS-R file but are present in the Dublin Bus app API response. The GTFS-R seems reliable for GoAhead routes though.

    Use the apisupport@nationaltransport.ie address for issues you find, the more people that are reporting them the more likely they are to do something about it.


  • Registered Users Posts: 2,562 ✭✭✭harringtonp


    smackyB wrote: »
    Not sure if you know but it's meant to be used in conjunction with the GTFS file - the GTFS-R feed just tells you about trips in progress and how to adjust the timings for those trips in the GTFS file.

    I've found it patchy so far, had quite a few reports from users of my app saying the times are off when they compare it to the official Dublin Bus app. From looking at the raw data I've seen quite a few cases where Dublin Bus trips don't show up in the GTFS-R file but are present in the Dublin Bus app API response. The GTFS-R seems reliable for GoAhead routes though.

    Use the apisupport@nationaltransport.ie address for issues you find, the more people that are reporting them the more likely they are to do something about it.

    Yes there is good RTD coverage for Dublin bus during the days but then once you get to early evening the number of routes covered drop way off. The other 3 operators covered don't show this trend. I've asked about this, got no response yet but I get the impression it is an improving situation.

    I suspect there own app is not reading directly from the feed but somewhere earlier in the pipeline.

    On a different note I noticed that route numbers can sometimes be different in their published data to what the app shows. For example on the Blanchardstown to Tallaght route their app lists a 76a but in the data it shows as 761. Anyone know what is going on here, has there been a recent route name change ?


  • Registered Users Posts: 746 ✭✭✭smackyB


    Yes there is good RTD coverage for Dublin bus during the days but then once you get to early evening the number of routes covered drop way off. The other 3 operators covered don't show this trend. I've asked about this, got no response yet but I get the impression it is an improving situation.

    I suspect there own app is not reading directly from the feed but somewhere earlier in the pipeline.

    On a different note I noticed that route numbers can sometimes be different in their published data to what the app shows. For example on the Blanchardstown to Tallaght route their app lists a 76a but in the data it shows as 761. Anyone know what is going on here, has there been a recent route name change ?

    Yeah I've asked them to fix this multiple times, GoAhead routes with letters are erroneously named with a corresponding number of the alphabet instead. I've had to manually fix this myself in the GTFS file.

    Good to know you've seen similar regarding patchy Dublin Bus data. It's frustrating TFI's own apps don't use the same datasource, otherwise I bet they would be much more proactive about fixing this.


  • Registered Users Posts: 2,562 ✭✭✭harringtonp


    I draw the line myself at doctoring GTFS data.

    The quality of GTFS headsign data has forced me to improve the implementation though. With Bus Eireann in particular, the headsign often contains the route name (e.g. Ennis Bus Station - Galway Bus Station) and sometimes buses stop short of the destination which can be misleading. So when appropriate I show the first and last stop name too.

    The TFI app loads data outside of GTFS when displaying stop names so you get for example the area along with the road name. They do make this available (Naptan, NPTG) for consumption but you would be working outside of GTFS and the implementation would only be good for TFI. In a good GTFS implementation (if anyone from TFI is reading this...) the stops.txt file would have everything you need. They only use stop_name but ideally would use the stop_code and stop_desc fields too.

    Pity they don't make the vehicle position feed available. There's not much that can go wrong with this as a timestamp and lat/lng are provided. And as long as users can see where the bus is, wayward predictions don't matter so much.


  • Advertisement
  • Registered Users Posts: 78,218 ✭✭✭✭Victor


    On a different note I noticed that route numbers can sometimes be different in their published data to what the app shows. For example on the Blanchardstown to Tallaght route their app lists a 76a but in the data it shows as 761. Anyone know what is going on here, has there been a recent route name change ?
    On the timetable sheets, these routes have been renumbered.


  • Registered Users Posts: 746 ✭✭✭smackyB


    Victor wrote: »
    On the timetable sheets, these routes have been renumbered.

    No sign of that on GoAhead's website, where have you seen that?


  • Registered Users Posts: 746 ✭✭✭smackyB


    Added an update to my apps to show which buses are being tracked in the GTFS-R system (grey is a non-realtime bus). Very poor coverage this morning, only 3/17 results are real-time :(

    Screen-Shot-2020-10-19-at-10-32-35-AM.png


  • Registered Users Posts: 2,562 ✭✭✭harringtonp


    None at the moment realtime showing on mine

    But it seems to fall out most evenings


  • Registered Users Posts: 2,562 ✭✭✭harringtonp


    Where as Go Ahead is fine


  • Registered Users Posts: 20 Capwell


    Thanks for your reply. I assumed it might work similarly to the old one. The GTFS data spec has the ability to include the arrival time but I guess it depends on the transport authority's implementation. I had the old one integrated into my home automation system to display it on a wall mounted screen (very popular with the family). So I thought I could just drop in the new version, but not so it seems. I have done a good bit of work to build a new version using all the GTFS static data which wasn't needed in the old version.

    Does anyone have a full decode of the Service IDs, e.g. "2#1" looks like it is Mon to Fri. Dublin Bus was using "3#1" for the bank holiday. Thanks


  • Registered Users Posts: 78,218 ✭✭✭✭Victor


    Capwell wrote: »
    Does anyone have a full decode of the Service IDs, e.g. "2#1" looks like it is Mon to Fri. Dublin Bus was using "3#1" for the bank holiday. Thanks

    If this is the service profile (how soon it gets to each stop), it seems there are no less than 17 of them for different types of days and times of day.


  • Registered Users Posts: 2,562 ✭✭✭harringtonp


    Anyone seeing a lot of read time outs when polling the RTD URL ?


    WARNING [2020-10-28 09:58:10] Error reloading Trip Update https://gtfsr.transportforireland.ie/v1/ [Read timed out]
    WARNING [2020-10-28 09:58:45] Error reloading Trip Update https://gtfsr.transportforireland.ie/v1/ [Read timed out]
    WARNING [2020-10-28 10:00:27] Error reloading Trip Update https://gtfsr.transportforireland.ie/v1/ [Read timed out]
    WARNING [2020-10-28 10:02:46] Error reloading Trip Update https://gtfsr.transportforireland.ie/v1/ [Read timed out]
    WARNING [2020-10-28 10:03:21] Error reloading Trip Update https://gtfsr.transportforireland.ie/v1/ [Read timed out]
    WARNING [2020-10-28 10:08:10] Error reloading Trip Update https://gtfsr.transportforireland.ie/v1/ [Read timed out]
    WARNING [2020-10-28 10:08:45] Error reloading Trip Update https://gtfsr.transportforireland.ie/v1/ [Read timed out]
    WARNING [2020-10-28 10:12:11] Error reloading Trip Update https://gtfsr.transportforireland.ie/v1/ [Read timed out]
    WARNING [2020-10-28 10:13:19] Error reloading Trip Update https://gtfsr.transportforireland.ie/v1/ [Read timed out]
    WARNING [2020-10-28 10:16:35] Error reloading Trip Update https://gtfsr.transportforireland.ie/v1/ [Read timed out]


  • Registered Users Posts: 20 Capwell


    There seems to be no obvious linkage between the agency_id and the service_id or am I missing something. There seems to be no rule for selecting the service_id to be used for selecting the trips and then the stop_times. Dublin bus seems to use 1 set of IDs, e.g. 2#1, 1#1, 3#1 and Goahead is using a different set, e.g.10706, 10707 etc.

    Does anyone know the rules?


  • Registered Users Posts: 2,562 ✭✭✭harringtonp


    Capwell wrote: »
    There seems to be no obvious linkage between the agency_id and the service_id or am I missing something. There seems to be no rule for selecting the service_id to be used for selecting the trips and then the stop_times. Dublin bus seems to use 1 set of IDs, e.g. 2#1, 1#1, 3#1 and Goahead is using a different set, e.g.10706, 10707 etc.

    Does anyone know the rules?

    It's a lot more work than a straight forward api.

    Read the spec carefully at

    https://developers.google.com/transit/gtfs/reference

    For your purposes focus on the agency, routes, trip, calendar, stops and stop times tables.

    The information is spread across all these files and if you're loading them into equivalent db tables you'll be constructing a sql query across them


  • Advertisement
  • Registered Users Posts: 2,562 ✭✭✭harringtonp


    The GTFS RT Api has been down for me since 3pm yesterday. The call succeeds but there is no data.

    Anyone else seeing data ?


  • Registered Users Posts: 20 Capwell


    Hi, thanks for your reply. I have been able to build a version which works fine for me (still testing) but I still have that one issue about the service_id. A quick scan of the spec doesn't show which service_id to use for a specific agency, i.e. Dublin Bus or Go Ahead. My 2 local stops are 1 of each. So for now, I have hard wired the linkage based on:

    Dublin Bus
    Mon-Fri "2#1"
    Sat "1#1"
    Sun "3#1"

    Go Ahead
    Mon 10701
    Tue 10702
    Wed 10703
    Thu 10704
    Fri 10705
    Sat 10706
    Sun 10707

    I still don't have a solution for Bank Holidays, Easter, Christmas but I'll have a look at the spec in more detail later. The TFI app was not showing real time data yesterday and again this morning.


  • Registered Users Posts: 2,562 ✭✭✭harringtonp


    Hi,

    In trips.txt you have service_id and route_id along with other trip information

    In routes.txt you have route_id and agency_id along with other route information

    And in agency.txt you agency_id and agency_name (and other agency information)

    So you derive the agency name from the service id via an effective join of these 3 files.

    calendar_dates can remove specific days for a service and add days such as bank holidays to an existing service or a brand new service.


  • Registered Users Posts: 20 Capwell


    Thanks for the reply, it's given me an idea how to solve it based on the way I built it for my system, or later on, I may try a different approach.


  • Registered Users Posts: 2,562 ✭✭✭harringtonp


    It can be fun figuring this out but if you're looking for an easier way you're welcome to leverage of my backend. Send me the stop ids of the stops you are interested in and I will construct the URL(s) for you. Data is returned in JSON format


  • Registered Users Posts: 746 ✭✭✭smackyB


    The GTFS RT Api has been down for me since 3pm yesterday. The call succeeds but there is no data.

    Anyone else seeing data ?

    Same here :(


  • Registered Users Posts: 20 Capwell


    It can be fun figuring this out but if you're looking for an easier way you're welcome to leverage of my backend. Send me the stop ids of the stops you are interested in and I will construct the URL(s) for you. Data is returned in JSON format

    Hi, thanks for the offer but I have been able to work it out. I have built a 'bus sensor' which is integrated into my home automation system (Home Assistant running on a Raspberry Pi3), replicating the setup I had with the previous API. It will require more maintenance because of all the static reference data required and the Calendar information which will need to be refreshed at year end I suppose.


  • Registered Users Posts: 20 Capwell


    The new data for part of 2021 has been issued on the NTA web site. A lot of the route IDs have changed, e.g.
    route:

    '60-16-d12-1'
    is now
    '60-16-b12-1'

    anybody know why?


  • Registered Users Posts: 2,562 ✭✭✭harringtonp


    I don't but can tell you they were having problems with route ids. Sometimes, when unscheduled trips are added in GTFS RT they don't reference an existing route. Other times, when you see two added RT entries for a single trip the only difference between them is the route id. One of the route id's will reference routes.txt and the other won't. I highlighted this and unlike other issues, a case number was issued. If this is a side effect of a solution then great, I will have to check it out at some point.


  • Advertisement
  • Registered Users Posts: 2,562 ✭✭✭harringtonp


    Actually just looked back over examples I sent in and route ids which didn't reference routes.txt had d12 in them. Those which did had b12

    Are you taking your information solely from tripUpdates ? If so I'd double check that any route ids you use exist in routes.txt. To conform to the spec it should.


Advertisement