Defining Metrics and Key Performance Indicators

Cityscope calculates a host of metrics using the data it collects. This article defines these metrics and how they are calculated. It attempts, as much as possible, to use plain language and clear examples to help you understand every calculation. As always, you are always welcome to chat with us if you feel that you are missing information!

📘

How Cityscope collects data and calculates metrics

Cityscope collects data in hourly chunks.

Let's imagine there is a provider RIDE in the City of Paris. To collect data about RIDE, Cityscope will ping RIDE's API every hour, generally 10 to 20 minutes after the top of the hour (say, 10h20), requesting from the API all data from the previous hour (in this case, all device events between 9h to 10h). This data chunk includes all the devices that changed statuses in the past hour (was parked, went unavailable, was reserved by a user, was removed from the fleet, etc.)

Every metric is then calculated over these hourly chunks, using the last hourly chunk data collected, and previous hours if relevant to the calculation.

Some metrics are calculated based on the state of the system at the top of the hour (for example, fleet size is calculated based on all devices that are in the public right of way at the beginning of the hour), while other metrics are calculated using all events that took place over the hour (for example, trip counts calculated all trips taken over the hour).

The way specific metrics are calculated is described below.

📘

Aggregating metrics above an hour

By default in its Export, Report, and API functions, Cityscope averages all aggregations above an hour. In other words, if you ask Cityscope the daily Fleet size, it will take the 24 Fleet sizes it calculated at the top of the hour and average them. It will also do this for metrics like Trip count. If you ask Cityscope for the daily Trip count, it will return the average hourly Trip count. To get something like the total Trip count per day, currently its best to export your data by the hour and sum across the day yourself.

When computing averages across the entire city, we count all trips by all providers, and provide a daily average of that sum.

Example:

For Days 1,2,3

If Provider A has 10, 15, and 20 trips for each day, and Provider B has 12, 17, and 22 trips for each day, then the average/total number of trips across the entire city for each day is 22, 32, and 42. If we take the average over all days then the overall average is (22 + 32 + 42) / 3 = 32

Fleet Size and Availability

Fleet size is the total number of devices "on street" (in the public right of way), at the top of the hour

A device that is "on street" or in the public right of way is a device that is outside, in public, whether functioning or not. In the Mobility Data Standard, there are three statuses that can represent a device parked "on street": unavailable, reserved, and available. A device is added to fleet size when it has one of these statuses. It is removed when the provider endpoint sends a status of removed. If a device sends no update for over one week (168 hours), Cityscope assumes it has been removed from the public right of way.

For example, give Device X that sent a status of available

We calculate fleet size once per hour, at the beginning of the hour. We will also show average fleet sizes for a particular time period, which is simply the sum of the hourly fleet counts, divided by the total number of hours in the time period.

Fleet size can be calculated for any geographic level- the entire city, districts/subdistricts, zones, hexagons, or any other features. A device is counted as part of the fleet size if it is situated in the geographic level at the moment of the count.

Fleet availability is a ratio of total devices with MDS statuses "available" or "reserved" over the total fleet size, per hour

Those devices that are in good working order and either available for rental, or actually rented. It is the sum of all devices "available" and "reserved", divided by the total number of devices "available", "reserved", and "unavailable" (the fleet size).

The availability metric is calculated once per hour, at the same time that the fleet size is calculated.

Trip Counts

Cityscope bases the trip count calculation mainly on the /trips and /status_change endpoint. We take into account the trip_start and trip_end events as well as the unique trip id, duration and distance assigned to each trip. The system filters out automatically trips that are considered too long, meaning:

• longer than 48h for cars
• longer than 2h for mopeds
• longer than 12h for the rest
• distance longer than 300km for cars
• distance longer than 100km for the rest

There are several measures associated with trip counts.

Count of trips originating from is the number of trips that originated from the given geography, per hour

Count of trips ending in is the number of trips ending in the given geography, per hour

These numbers are likely going to be different, as the inflows and outflows of any given geography will vary. There will also be variance because trips may begin in one hour, and end in the next. In Cityscope, we default to showing you "Count Origin".

Trip counts are typically provided in the context of an hourlong period. In most cases, they represent an average hourly rate. To know the total number of trips taken for longer time periods, simply multiply the rate by the number of hours in the period (ie, to get the number of trips in a day, multiply by 24). This situation is present in the Export and Report process, though in Activity you the calculation will be done for you.

Count Origin and Count Destination can be calculated for any geography- the entire city, a sub-district or district, or even a custom zone.

Count of trips using road is the number of trips that traversed a given road, per hour.

We provide more information about how we build our street segment analysis using mid-trip telemetries here, but for now, a good definition of Count Riding is that it represents the number of trips using each segment each hour. It is only calculated for street segments, trying to calculate it over other types of geographies will produce an error.

Device Rotation

Device rotation is a ratio of Count of trips originating from over Fleet size per hour. In other words, it is the number of trips, per device, per day. We recommend analysing usage over a period of a day, as most vehicles will be used no more than once every few hours.

If you calculate device rotation for a period shorter than one day, we convert the ratio to a daily rate- in other words, the daily rotation as if the entire day was like the time period you requested.. If you calculate device rotation for a period longer than one day, we show an average of the daily rotations.

It is a measure of the productivity and utilization of the existing fleet. It can also be calculated by any geography- if you use a geography smaller than the entire city, it will be calculated based on the number of trips originating in that zone and the average fleet size of that zone.

Average device idle time is the average time, in seconds, between trips for all devices within the given geography. It captures the average duration devices tend to sit in the public right of way between uses.

Trip Distances and Durations

Distance of trips originating from is the average distance, in meters, of trips originating from the given geography, per hour

Distance of trips ending in is the average distance, in meters, of trips ending in the given geography, per hour

Duration of trips originating from is the average duration, in seconds, of trips originating from the given geography, per hour

Duration of trips ending in is the average duration, in seconds, of trips ending in the given geography, per hour

Generally, Trip Distance and Duration are provided directly in the MDS feed that Vianova receives from operators, so we do not calculate it, but rather simply present the average of all trips received in the period covered and the geography requested.

There are a few exceptions, however.

The distance between an origin and a destination can be reported two ways. The first is the crow-flies distance, which represents a straight line between the origin and destination (the shortest distance). The second reporting is a on-road distance, which represents the twists and turns that are necessary on the ground in order to get between the origin and destination. Many factors can contribute to the difference between crow-flies and on-road distances such as the complexity of the street grid, the location of infrastructure such as cycle paths, the weather, and even the price to rent devices.

If a provider is only sharing GBFS data, the feed does not include any distances, and so we calculate the crow-flies distance between where the device began the trip and where it ended. Comparing or averaging crow-flies and on-road distances together is a bad practice, because they are two different measures. In order to approximate the on-road distance when only crow-flies is available, Vianova multiplies all crow-flies distances by a factor of 1.5. This number reflects the most common difference between crow-flies and on-road distances and allows for more meaningful analytics, although it should be considered an estimate. As operators improve the distances they report, we can gradually replace all crow-flies distances with more accurate on-road conditions.

Compliance and regulation metrics

Count Infringement represents the number of infringing devices, or (in the case of a fleet based policy like caps and floors) whether there is a violation of the policy.

Regulated value is the count of the value under regulation. For "Cap", "Floor", and "Cap and Floor" regulations, the regulated value is the total fleet size across the entire geography. For "No go" and "No parking" regulations, the regulated value is the total number of infringing devices within the regulations' geographies.

Given all the infringements currently infringing at a given hour, average infringement duration is the average time since these infringements began, in hours. The metric captures the average length of infringements (the time from the beginning of an infringement to when it is resolved)

Unique infringements represent the count of infringements that have begun per hour. In other words, it is the newest infringement each hour.

These metrics may become more clear when you examine them in the context of specific policies.

Cap and Floor Violations

A violation is triggered on a cap or floor policy when the fleet size either exceeds an imposed cap or fails to meet an imposed floor. Caps and floors can be calculated citywide, or in a specific zone, and you can read more about the strategy behind fleet size policies here. The size of the fleet is calculated in the same manner as described above, so all vehicles in the right-of-way are included.

Caps and floor violations are calculated once per hour, and an operator is either "compliant" or "not compliant". The intensity of the "not compliant" is unimportant, the violation is treated the same whether an operator is over the cap by 10 devices or 100. In the export, count_infringement = 1 when there is a violation on the hour, and = 0 when there is no violation.

A compliance rate can be calculated by averaging this compliance metric over time. It is the inversion of "Count infringement", ie if an operator was compliant for 9 hours and not compliant for 1 hour, they would have a 90% compliance rate for this 10 hour period.

You can also use the regulation value metric to visualize the fleet size compared to the cap and floor

No Parking Violations

No parking violations are calculated by counting the number of devices parked in a no-parking zone each hour. A device may only be in violation of one zone per hour (ie the zone it is sitting in during the count, at the top of the hour). Because the typical units of violations are 1 violation per hour, over the course of the day, a single incorrectly parked device could accrue 24 violations, one per hour.

The Infringement Count calculates the average number of devices in violation of a no-parking zone (or of all no-parking zones) per hour. In other words, if you were to visit that particular no-parking zone at any hour in the period, you would expect to see X devices in violation.

The Unique Infringement Count consolidates violations of the same device in the same zone into a single violation. So one device incorrectly parked for 10 hours would be represented as 1 unique violation. Devices are counted exactly once, in the hour when they begin their violation. In other words, this metric measures the rate of new violations occurring each hour.

Infringement Hours or Average Infringement Duration calculates the average amount of time that a single device is in violation of a single policy. In other words, if violations are occurring, this metric indicates for how long they are occurring, on average.

For more information about parking policies, visit this page.

No-Go and Low-Speed Violations

No-Go zones are meant to prevent all riding and parking behavior in a particular zone. No-Go zones count any time a vehicle enters the zone, determined with any status update (trip trajectory, status change, etc.). A device can violate a no-go zone once per hour, either by passing through it, or by parking in it. Because not every company shares information about mid-trip telemetries, it is likely that the number in Cityscope represents an undercount.

Similar to No Parking Violations, No-Go Violations can either be calculated once per hour (Count Infringement), or exactly once, regardless of the duration (Unique Count Infringement).

Though Low-Speed Zones are communicated to Mobility Operators, Cityscope does not presently calculate any violations of low-speed zones. This is primarily related to the lack of high quality mid-trip telemetry data, making it difficult to remotely ascertain vehicle speed. For more information, read this article on riding policies.

Max Parked and Max Unavailable Violations

Maximum Parked and Maximum Unavailable refer to two different types of policies:

Device Based Max Parked and Max Unavailable track the duration of a period in a specific status, either "available" (but not rented) and not moving, or "unavailable". These policies are in place to encourage operators to rebalance devices that are in the right-of-way but are not productive.

These policies are calculated once per hour, and a device begins violating the policies once the duration of its status exceeds the threshold. For example, if a city has a policy where a device can spend no more than 24 hours in an "unavailable" status, the device will begin generating one violation per hour beginning in hour 25. These policies operate similarly to No Parking policies, and can similarly be calculated once per hour (Count Infringement), or exactly once, regardless of the duration (Unique Count Infringement).

Fleet Based Max Parked and Max Unavailable tracks the overall number of devices in an operators fleet with a particular status. For example, the city may indicate that no more than 100 devices can be "unavailable" at any given time. These versions of the policies operate similar to caps and floors, and can similarly be viewed in terms of a compliance score. You can "publish" the policy and notify operators.