Request Upload

Generate a new upload request for a stream.

The response will contain a unique identifier for the upload request, which allows
polling for status updates on the upload.

The response also contains the upload_s3_url which can be used to upload the data via
a PUT request.

Passing dry_run=true to the request will return the upload request where uploaded
files will only be validated for correctness, without insertion into the database.

For now, we have pretty strong requirements to upload a file as a stream, and we still have some limitations before building a more flexible system ( Confluence documentation here ):

Geographic data :

• GeoJson

  • Must follow the standards : https://geojson.org/
  • It can contain multiple properties, but it can’t deal with encapsulated properties for now, with multiple levels. The encapsulated properties will be automatically deleted from the source data.
    • OK
      JSON

      {
        "type": "Feature",
        "geometry": {
          "type": "Point",
          "coordinates": [125.6, 10.1]
        },
        "properties": {
          "name": "Dinagat Islands"
        }
      }
      

    • NOT OK
      JSON

      {
        "type": "Feature",
        "geometry": {
          "type": "Point",
          "coordinates": [125.6, 10.1]
        },
        "properties": {
          "name": "Dinagat Islands",
          "encapsulated_property": {
            "property": "value"
          }
        }
      }
      

• CSV

  • It’s mandatory it contains a geometry information which can be found in 4 different ways :
    • A column named geometry containing the feature geography in geojson format geometry
      JSON

      {
        "type": "Point",
        "coordinates": [125.6, 10.1]   
      }
      

    • 2 columns named latitude and longitude which will automatically build Point features
    • 1 column named h3_token containing the h3 token, following the h3 convention : https://www.uber.com/en-FR/blog/h3/
    • 4 columns named like below which will automatically build LineString feature. Warning : The lines created will be straight line, without caring about the streets shapes
      • latitude_<something_1>
      • longitude_<something_1>
      • latitude_<something_2>
      • longitude_<something_2>

Time data :

• The geojson or csv files can contain a time dimension, really interesting for a deeper analysis on Cityscope, as a time series stream.
• The datetime column must respect this format : 2022-05-04 or 2022-05-04T22:00:00
• If multiple datetime columns are in the file, the loading script will randomly take the first one it sees : It might be a good idea to make sure you only keep the datetime column you’re interested in