In this blog post you will learn how AWS Amplify helps to easily integrate AWS user management and storage into a frontend application. We will use Kepler.gl as an exemplary frontend application. Kepler.gl is a very interesting and high-performance application for map visualizations and geospatial analysis. To enable secure storage of both private and public Kepler.gl maps on AWS, we implemented an AWS Account Provider (in addition to Dropbox and Carto.com) based on AWS Amplify. We contributed the solution to the Kepler.gl Open Source community (currently it's an open pull request).
tl;dr: Show me the demo
We are a team of Data Engineers, GIS and ML experts developing a Geodata platform (www.deepatlas.io) and customer specific location-based services for big companies in the automotive and energy sectors or for public transport.
We are dedicated to AWS as a cloud platform which enables us to easily develop, deploy and scale our solutions.
Our solutions contain e.g.
- the most complete map for charging stations in Germany:
- intelligent routing algorithms for on-street parking spots
- a scalable, cost efficient GIS platform (part of it is based on Kepler.gl)
- an API with interesting features for training geospatial machine learning models:
Kepler.gl is “a powerful open source geospatial analysis tool for large-scale datasets”. [https://kepler.gl/]
It is performing the analysis and generating the visualizations just in your browser. It is fascinating to see how much data can be visualized simultaneously, which is often the bottleneck in standard map visualization tools. To achieve this, it makes heavy use of your graphics card which results also in nice 3D visualizations.
Kepler.gl in action at one of our projects
Because all the data is processed on client side, you have the option to export the whole map into a zip file and send it to your colleague, which is fine for some use cases. However in this post we will show you how you can store the data in a cloud backend and share, access and manage your datasets and maps more easily and securely.
AWS Amplify and it’s managed AWS service
“AWS Amplify is a set of products and tools that enables mobile and front-end web developers to build and deploy secure, scalable full stack applications, powered by AWS.” [https://aws.amazon.com/amplify/]
Diagram: Illustration of AWS infrastructure for a Kepler.gl storage provider with AWS Amplify
With the AWS Amplify CLI it’s super easy to setup the AWS infrastructure for your application. To create an AWS infrastructure for our connection between AWS and Kepler.gl - as depicted in the diagram - we just had to run the following commands:
$ amplify add auth
$ amplify add storage
The commands create a new Cloudformation stack, a Cognito user pool and a S3 bucket and connected both via IAM roles.
How Kepler.gl and AWS Amplify fit together
In the current demo deployment of Kepler.gl you can store your maps on Dropbox our Carto.com. These, however, are third party providers you don't have fully control over. Instead, we wanted to store our maps on our own infrastructure that is maintained by us and uses our own user management. In addition to that, we wanted to test out AWS Amplify. AWS is a natural choice for us anyway as our cloud infrastructure is mainly built on AWS and we have a lot of experience with different AWS services.
The following image shows how the integration of our "AWS account provider" for Kepler.gl looks like.
Screenshot: Login to your AWS Cognito account in Kepler.gl
Now to the details of our integration of Kepler.gl and AWS Amplify. If you aren't interested in the details, you can directly jump to the demo section. The code is available in our pull request for Kepler.gl
When a user tries to login, a new browser window with a React component (AwsLogin) opens. It uses the AmplifyAuthenticator component and sends an event back to the initiator if login was successful.
Our AwsProvider component listens to the login event and handles user login with the help of the Amplify Auth API.
After successful login, the user has access to the maps saved in private and public S3 storage. Amplify provides a Storage API to access S3. This API provides different privacy settings for data access which support all Kepler.gl features which are available for the existing providers.
Private, public and shared maps
A Kepler.gl storage provider needs to implement the concept of public and private maps as well as the possibility to share a map via an URL.
When the user wants to share a map via URL with someone else, we first upload the map by setting the Amplify storage level to protected in the Storage.put() method. This results in the map to be saved to the protected "subfolder" on AWS S3. Additionally to get a sharing URL we call Storage.get() with download=false and expires= expiry_time_seconds. The maximal expiration time for this URL is by default one hour in AWS. It's possible to lower the expiration time in seconds in the code.
Screenshot: How to create a link to share a map with other users
When the user wants to save a map, we save it to her private S3 storage by setting the Amplify storage level to private in the Storage.put() method. There is currently no possibility to save to the public storage from the Kepler.gl user interface. On the other hand, it is also not possible to send the Storage.put() requests to the public AWS S3 folder. Saving maps publicly has therefore to be done differently, e.g. copying the map files to the public S3 folder in the AWS console.
With the Kepler.gl map file which contains the map raw data and it's config for visualization we store a meta data file which contains e.g. the description of the map and a thumbnail.
Screenshot: Save a Kepler.gl map with description to AWS S3
For those who are interested in the security settings on AWS, we created a Gist containing the different policies for the Cognito IAM roles (private, protected, public): https://gist.github.com/LHaferkamp/ee232f78c57ec08e05894afadd893cc7